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FOREWORD 


As  the  Bureau  of  Quality  Assurance  began  implementation  of  the  PSRO  program 
in  hospitals,  it  soon  became  evident  that  the  future  would  require  PSROs  to 
be  actively  involved  in  ambulatory  medical  care  quality  assurance.  Since 
the  state  of  the  art  of  ambulatory  care  review  is  far  less  advanced  than 
that  of  hospital  review,  we  realized  that  it  would  be  necessary  to  develop 
a  sound  basis  for  the  careful,  gradual  movement  of  PSROs  into  ambulatory 
care  quality  assurance. 

This  three-volume  report  is  the  result  of  the  work  completed  by  Health  Care 
Management  Systems,  Inc.,  in  one  of  the  first  major  Bureau  contracts  to 
acquire  and  document  knowledge  of  presently  operational  ambulatory  care 
quality  assurance  projects.    The  purpose  of  this  project  was  to  systematically 
assess  and  document  existing  activities  in  a  variety  of  health  delivery  set- 
tings across  the  country.    During  the  course  of  the  project,  a  generic 
quality  assurance  model  was  developed  for  use  in  the  design,  implementation, 
and  assessment  of  ambulatory  care  review  systems.    This  model  is  now  under 
refinement  and  is  being  applied  in  the  Bureau's  cooperative  ambulatory  care 
quality  assurance  demonstration  project.    It  represents  a  significant 
advance  in  our  ability  to  design  and  assess  such  systems. 

This  three-part  report,  which  includes  a  description  of  the  generic  model 
and  study  findings,  description  of  each  of  the  27  ambulatory  quality 
assurance  systems  included  in  the  survey,  and  a  bibliographic  index  that 
has  been  enriched  by  selected  abstracts,  should  provide  the  reader  with  a 
concise  overview  of  the  state  of  the  art.    The  results  of  this  project  are 
an  important  part  of  the  foundation  upon  which  to  build  well-balanced  and 
meaningful  PSRO  participation  in  ambulatory  care  quality  assurance.  The 
report  also  reflects  the  investigators'  concern  with  ambulatory  quality 
assurance  outside  the  scope  of  PSRO. 

The  authors  of  the  report  developed  their  material  in  a  relatively  short 
time  to  meet  the  need  for  fundamental  knowledge  of  existing  quality 
assurance  programs  to  support  the  implementation  of  voluntary  systems  and 
governmental  regulatory  programs.    It  is  hoped  that  any  limitations  in 
the  material  presented  herein  will  be  corrected  through  additional  research 
and  careful  experimentation  and  demonstration  of  new  ambulatory  quality 
assurance  methods  and  organization. 

We  believe  that  this  report  will  be  valuable  to  those  within  and  outside 
the  Federal  Government  who  are  interested  in  establishing  effective  quality 
assurance  systems  in  ambulatory  medical  care.    We  are  pleased  to  have 
provided  partial  funding  to  the  project  and  the  preparation  of  this  mono- 
graph and  hope  that  it  will  be  useful  to  all  who  are  now  engaged  in 
activities  leading  to  improved  quality  in  ambulatory  medical  care. 


Michael  J.  Goran,  M.D. 
Director 

Bureau  of  Quality  Assurance 
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ABSTRACT 


That  quality  care  assurance  is  in  a  state  of  flux  is  attested  to  by 
current  literature  in  the  field  and  is  substantiated  by  the  Ambulatory 
Care  Qual ity  Assurance  Project  (ACQAP)  report.    This  nationwide  survey 
was  undertaken  to  reveal  the  state  of  the  art  in  the  United  States 
today.    During  a  study  of  more  than  two  dozen  sites  over  a  wide  range 
of  geographic  locations,  the  ACQAP  produced  a  set  of  basic  program 
procedures,  a  scoring  method,  and  what  appears  to  be  a  sound  basis 
for  planning,  operating  and  assessing  quality  care  assurance  programs. 
The  literature  relative  to  quality  assurance  in  ambulatory  care  was 
investigated  and  a  collection  of  the  better  references  accumulated. 

Volume  I  of  the  three-volume  report  contains  a  detailed  description 
of  the  ACQAP  Model,  its  scoring  system,  and  its  findings.    The  data 
collection  instrument,  a  226-item  open-ended  questionaire  used  in 
the  study,  is  contained  in  an  appendix. 

Volume  II  contains  detailed  descriptions  of  the  27  programs  reviewed 
in  the  course  of  the  ACQAP  accompanied  by  a  compact  tabulation  of 
their  key  characteristics  for  quick  reference,  so  that  the  reader 
may  compare  the  study  sites  with  his  own. 

Volume  III  is  a  bibliography  containing  a  general  list  of  the  articles 
and  other  references  found  to  be  most  useful  to  the  study.    The  list 
is  duplicated  under  several  topic  headings  for  the  convenience  of  the 
reader  interested  in  a  particular  aspect  of  quality  assurance  in 
ambulatory  medical  care.    A  number  of  the  references  have  been 
abstracted  and  the  volume  also  contains  a  narrative  review  of  the 
1 iterature. 
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INTRODUCTION 


Volume  II  describes  the  specific  ambulatory  quality  review 
activities  and  program  methodologies  used  at  26  sites.    The  survey  of 
sites  focused  on  collection  of  data  which  would  describe  not  only  the 
methods  used  in  various  quality  assurance  programs,  but  their  corres- 
ponding administrative  structure,  resource  allocation,  and  program 
management,  all  of  which  may  have  impact  on  a  quality  assurance  program. 
The  descriptions,  which  strive  to  reflect  the  programs  accurately,  are 
based  upon  a  detailed  data  collection  instrument  used  both  to  collect 
documentation  before  site  visits  and  to  serve  as  a  basis  for  interviews. 

Study  investigators  attempted  to  distinguish  between  quality 
assurance  activities  as  planned  and  activities  as  ultimately  imple- 
mented.   The  sites  surveyed  provide  a  diverse  sample  of  programs,  as 
reflected  by  variations  in  the  length,  organization  and  content  of 
individual  descriptions.    The  reader  should  appreciate  that  providing 
a  total  picture  of  a  program,  while  honoring  the  need  for  succinctness 
and  brevity,  was  difficult.    Signed  approval  for  the  program  descriptions 
was  obtained  from  all  sites. 

Information  presented  here  was  compiled  to  answer  basic  questions 
about  various  aspects  of  quality  assurance  programs  and  to  allow  the 
reader  to  match  his  own  setting's  characteristics  with  those  of  one  or 
more  of  the  sites  surveyed.    Comparison  of  sites  and  programs  is 
facilitated  by  a  chart,  "Summary  of  Review  System  Characteristics," 
which  precedes  the  descriptions  and  briefly  summarizes  eight  character- 
istics.   The  chart  should  enable  the  reader  to  make  quick  comparisons 
among  sites  or  to  select  sites  similar  to  his  own  without  reading  all 
the  material.    Questions  regarding  individual  sites  not  answered  in 
this  text  may  be  addressed  directly  to  Health  Care  Management  Systems 
(HCMS)  staff  or  to  personnel  at  the  site. 
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SUMMARY     OF     REVIEW     SYSTEM     CH  A  R  ACTERISTICS 


The  following  chart  briefly  describes  eight  selected  characteristics 
of  every  site  visited  and  surveyed.    The  eight  characteristics  represent 
aspects  of  the  review  systems  described  which  were  identified  by  investi- 
gators as  vital  to  a  quick  overview  and  well-rounded  impression  of  individual 
quality  assurance  programs  and  the  surveyed  sites  as  a  whole.  Characteristics 
are  as  follows: 

Administrative  Structure:    Outlines  organization  responsibilities, 
decision  making  framework,  and  structural  scheme  used  to  manage 
quality  review  system. 

Topic  Identification:    Personnel  and  method  used  to  decide  which 
segment  of  medical  care  and  the  organization  the  review  system 
will  examine. 

Case  Selection:    Methods  and  procedures  for  choosing  cases  to  review. 

Data  Source:    Documents  used  to  collect  data  for  review:  Medical 
Record,  Problem-Oriented  Medical  Record,  Patient  Questionnaire, 
Encounter  Form,  Billing  Claims. 

Criteria  and  Guidelines  Used  in  Review:    Identifies  what  types  of 
criteria  (explicit,  implicit,  documented)  are  used,  and  their 
appl ication. 

Review  Procedures:    Describes  who  conducts  review  and  what  data  the 
reviewer  examines  during  the  review  process. 

Decision  Making:    Describes  who  makes  decisions  and  what  types  of 
decisions  are  made  for  review  of  specific  cases. 

Feedback  Procedures:    Describes  types  of  activity  and  personnel 
responsible  for  implementing  feedback  based  on  review  decisions. 

The  chart  can  be  used  to  form  a  quick  impression  of  a  site's  program,  to 
compare  sites,  and  to  generalize  information  about  all  sites  under  one 
characteristic.    The  explanatory  statements  under  each  characteristic  may 
seem  similar  for  different  sites,  but  a  close  reading  of  all  statements  will 
reveal  a  diversity  of  methods  and  activities.    Examination  of  the  chart 
should  provide  the  reader  with  a  background  of  the  similarities  and  differences 
in  quality  review  methods  and  programs  as  surveyed  by  this  project. 
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LIST  OF  SITES  VISITED*,  KEY  TO  ABBREVIATED  NAMES ,  AND  CLASSIFICATION  BY 


SOURCE  OF  FUNDING 


1 . 

CMP 

2. 

GHCC 

3. 

NCHPP 

4. 

MTHP 

5. 

SLPMC 

6. 

HCHP 

7. 

GHA 

8. 

KNC 

9. 

HMO  I 

10. 

NEVHC 

1 1 

II* 

DMC 

LSI  1  \J 

12 

CHSD 

1  3 

MLK 

14 

HfNHf 

15 

HN 

16 

VFHC 

1  7 

X  1  IO 

18 

FHCP 

1  9 

1  J  • 

NfHP 

20. 

SHA 

21. 

BSC 

22. 

UPRO 

23. 

NMPSRO 

24. 

BIACP 

25. 

MFMC 

26. 

CACS 

27. 

DPMR 

COLUMBIA  MEDICAL  PLAN  (prepaid) 

GIVEN  HEALTH  CARE  CENTER  (fee  for  service) 

NORTH  CENTRAL  HEALTH  PROTECTION  PLAN  (fee  for  service) 

MATTHEW  THORNTON  HEALTH  PLAN  (fee  for  service) 

ST.  LOUIS  PARK  MEDICAL  CENTER  (fee  for  service) 

HARVARD  COMMUNITY  HEALTH  PLAN  (prepaid) 

GROUP  HEALTH  ASSOCIATION,  INC.  (prepaid) 

KAISER  PERMANENTE  MEDICAL  CARE  PROGRAM,  NORTHERN  CALIFORNIA  (prepaid) 

HMO,  INTERNATIONAL  (prepaid) 

NORTHEAST  VALLEY  HEALTH  CORPORATION  (prepaid) 

DOWNSTATE  MEDICAL  CENTER  (PEDIATRIC  AMBULATORY  SERVICE)  (grants/local 
tax  supported) 

COMPREHENSIVE  HEALTH  SERVICES,  DETROIT  (grants/local  tax  supported) 

MARTIN  LUTHER  KING,  JR.  HEALTH  CENTER  (grants/local  tax  supported) 

HARRIS  COUNTY  NEIGHBORHOOD  HEALTH  CLINICS  (grants/local  tax  supported) 

HOUGH  -  NORWOOD  FAMILY  HEALTH  CENTER  (grants/local  tax  supported) 

VINELAND  FAMILY  HEALTH  CENTER  (grants/local  tax  supported) 

INDIAN  HEALTH  SERVICE  (TUCSON,  ARIZONA)  (direct  government  support) 

FLORIDA  HEALTH  CARE  PLAN  (no  description)  (qualified  HMOs) 

NORTH  COMMUNITIES  HEALTH  PLAN  (qualified  HMOs) 

SOUND  HEALTH  ASSOCIATION  (Qualified  HMOs) 

BLUE  SHIELD  OF  CALIFORNIA  (PSRO) 

UTAH  PROFESSIONAL  REVIEW  ORGANIZATION  (PSRO) 

NEW  MEXICO  PROFESSIONAL  REVIEW  ORGANIZATION  (PSRO) 

BETH  ISRAEL  AMBULATORY  CARE  PROJECT  (research) 

MULTNOMAH  FOUNDATION  FOR  MEDICAL  CARE  (research) 

CONNECTICUT  AMBULATORY  CARE  STUDY  (research) 

DEPARTMENT  OF  PHYSICAL  MEDICINE  &  REHABILITATION  (UNIVERSITY  OF 
MINNESOTA)  (grants/local  tax  supported) 


*Sites  are  listed  in  order  of  appearance  on  characteristics  chart  as  in  Table 
of  Contents. 
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COLUMBIA  MEDICAL  PLAN 
Columbia,  Maryland 


SECTION  I 
INTRODUCTION 


The  Columbia  Medical  Plan  (CMP)  was  formed  in  1969  through  the  joint  acti- 
vities of  Connecticut  General  Life  Insurance  Company  (CG)  and  the  Johns  Hopkins 
University  in  order  to  jointly  develop  and  operate  a  prepaid  health  care  system 
for  residents  of  Columbia,  Maryland.  This  mul tispecialty  group  practice  serves 
approximately  20,000  enrollees  in  Columbia,  Maryland,  and  the  surrounding  area. 
The  Columbia  Division  of  the  Johns  Hopkins  Health  Service  Research  and  Develop- 
ment Center  (JHHSRDC)  conducted  quality  assurance  research  at  the  Columbia 
Medical  Plan.  Research  was  funded  as  an  Experimental  Medical  Care  Review 
Organization  (EMCRO)  by  the  National  Center  for  Health  Services  Research. 

The  Columbia  Division  of  JHHSRDC  had  been  inaugurated  at  CMP  prior  to  the 
EMCRO  grant,  and  the  quality  assurance  research  was  only  part  of  a  larger  research 
enterprise.    The  association  with  Johns  Hopkins  HSR&D  Center  is  critical  for  CMP 
in  accomplishing  the  goal  of  maintaining  a  close  relationship  between  academic 
activities  and  an  operational  health  care  plan.    CMP  is  viewed  as  an  operational 
test  site  for  health  services  research  issues. 

The  EMCRO  project  was  designed  to  develop  and  test  an  outcome-based  metho- 
dology for  quality  assurance  in  primary  care.    The  program  was  included  in  this 
study  because  of  the  unique  opportunity  to  examine  an  interface  between  outcome- 
based  quality  assurance  research  and  the  operations  of  a  primary  care  health 
plan. 
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SECTION  II 

QUALITY     ASSURANCE  PROGRAM 


The  Columbia  Medical  Plan  -  Experimental  Medical  Care  Review  Organization 
(CMP-EMCRO)  quality  review  efforts  combine  the  implementation  of  certain  re- 
search studies  in  an  operational  delivery  system  that  has  the  potential  to 
implement  procedures  based  on  study  results.    Unlike  other  quality  assurance 
programs,  CMP-EMCRO  documentation  reflected  an  existing  and  operational 
rather  than  a  planned  program.    Therefore,  this  section  will  describe  opera- 
tional quality  assurance  studies  conducted  by  the  CMP-EMCRO  staff.    The  descrip- 
tion will  not  present  study  results,  but  only  quality  assurance  methodologic 
information.    The  following  discussion  is  divided  into  three  major  areas: 

1.  General  view  of  the  EMCRO  research  project  and  methodology 

2.  Descriptions  of  the  studies  implemented,  including  a  pilot 
study,  Round  I  and  II  studies  in  accessibility,  problem-status 
outcome  assessment  and  the  study  of  telephone  care 

3.  Use  of  the  CMP-EMCRO  study  results  as  a  basis  for  implementing 
administrative  and  organizational  changes  at  CMP 


BACKGROUND 

Development  of  the  program  was  stimulated  by  three  sources: 

1.  The  availability  of  funds  from  the  National  Center  for  Health  Services 
Research  for  research  in  quality  assurance  methods 

2.  Availability  of  an  operational  and  research  setting,  Columbia  Medical 
Plan 

3.  Existing  relationship  between  JHU  and  CMP,  and  the  accompanying  re- 
search and  medical  expertise 

In  February,  1976,  the  CMP-EMCRO  staff  included: 

1 .  Research  Program  Manager 

2.  Research  Associate 

3.  Biostatistician  (part-time) 

4.  Two  Research  Assistants 

5.  Medical  Record  Data  Clerk  (part-time) 

6.  Secretary  (part-time) 

The  central  feature  of  the  CMP-EMCRO  project  is  outcome  assessment  of  am- 
bulatory care  as  a  means  of  determining  the  quality  of  care  within  a  particular 
delivery  setting.  The  CMP-EMCRO  project  attempts  to  conduct  rigorous  research 
closely  linked  with  operational  medical  care  delivery.  The  outcome  assessment 
methodology  is  designed  not  only  to  develop  a  method  for  determining  the  level 
of  quality,  but  also  to  identify  deficiencies  in  the  delivery  of  medical  care 
that  result  in  substandard  outcome. 
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The  CMP-EMCRO  incorporates  a  specific  research  strategy  to  develop  mea- 
sures of  quality  appropriate  for  an  HMO  and  to  test  their  utility  by  demon- 
strating that  their  application  can  lead  to  identification  of  correctable 
deficiencies  in  the  delivery  of  care. 

The  CMP-EMCRO  approach  to  quality  assessment  assumes  "that  the  develop- 
ment of  a  meaningful  system  can  best  occur  in  a  stepwise,  incremental 
fashion.    In  other  words,  that  quality  assurance  occurs  in  a  series  of  rounds 
when  data  are  gathered,  instruments  and  measures  tested,  results  reported  and 
the  utility  of  the  measures  for  identifying  problems  in  delivery  of  care 
examined."    In  developing  an  approach  to  measuring  quality  of  care,  CMP-EMCRO 
investigators  chose  to  measure  variables  related  to  accessibility  of  care  and 
the  patient's  problem  status  outcome  as  indication  of  quality.    To  determine 
reasons  for  poor  access  or  substandard  outcome,  medical  records  for  the  study 
population  are  reviewed. 

In  terms  of  specific  quality  assurance  studies  conducted  by  the  CMP-EMCRO 
there  have  been  several  areas  of  investigation: 

1.  Population-based  accessibility  studies 

2.  Development  and  testing  of  patient-reported  problem  status 
as  an  indicator  of  quality 

3.  Determination  of  medical  care  process  as  a  search  for  reasons 
for  substandard  outcome  to  test  the  quality  assurance  potential 
of  an  outcome  strategy 

4.  Assessment  of  process  of  medical  care  for  common  conditions  as 
indicators  of  quality 

5.  Quality  of  telephone  care  by  nonphysician  personnel 

The  first  area  of  research  addressed  by  CMP-EMCRO  was  the  development  of 
specific  research  methods  for  each  study  area  noted  above.    Four  major  enrol - 
lee  questionnaires  were  administered:    a  pilot  test  in  April,  1973;  EMCRO 
Round  I  between  October  1  and  17,  1973;  EMCRO  Round  II,  May  22  to  June  13,  1974; 
EMCRO  Round  III,  October  17  to  31,  1975.*    In  addition,  a  study  of  the  quality 
of  telephone  care  by  nonphysician  personnel  was  conducted  in  November,  1974. 

Accessibility  of  CMP  services  to  the  enrolled  population  was  the  first 
area  studied  in  a  pretest  before  the  actual  funding  of  EMCRO.    After  developing 
the  patient  questionnaire,  staff  mailed  questionnaires  to  a  random  sample  of 
10%  of  the  enrollees  in  April,  1973.    The  questionnaire  collected  information 
in  three  areas: 

1.  Persons  with  problems  during  a  specified  period  of  time  who 
sought  and  received  care 

2.  Persons  with  problems  who  sought  care,  but  did  not  receive  it 

3.  Persons  who  did  not  seek  care 


*  Since  the  results  of  the  third  EMCRO  study  were  not  available  at  our  visit, 
that  study  is  not  included  in  this  description 
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The  CMP-EMCRO  staff  considered  this  sample  a  pilot  study  in  terms  of 
measuring  enrollee  response  (959  of  1339  enrollees  returned  questionnaires), 
discovering  the  extent  of  accessibility  problems,  and  collecting  initial 
data  for  future  comparative  studies  of  accessibility. 

Rounds  I  and  II  under  EMCRO  (October,  1973,  and  June,  1974)  included 
accessibility  studies,  problem  status  outcome  determinations  and  reviews 
of  medical  care  process,  to  identify  reasons  for  substandard  outcome. 


Round  I 

The  Round  I  visit  questionnaire  was  designed  to  measure  and  assess 
access  to  care  of  a  sample  of  patients  with  a  medical  problem.  CMP-EMCRO 
personnel  emphasized  that  a  large  part  of  Round  I  was  developmental  and  was 
an  attempt  to  assess  certain  quality  assurance  methodological  issues  in 
order  to  insure  proper  questions  would  be  addressed  in  subsequent  rounds. 

The  Round  I  data  collection  included  two  questionnaires,  the  first 
administered  to  patients  during  a  visit  and  the  second  mailed  to  them  approxi- 
mately one  month  after  the  visit.    The  first  questionnaire  was  designed  to 
collect  information  on  the  effort  involved  in  arranging  for  and  receiving 
care  (access)  and  the  patient-reported  problem  status  at  the  time  of  visit. 
The  second  questionnaire  was  designed  to  determine  outcome  by  measuring  a 
patient's  problem  status  one  month  after  the  encounter. 

Concurrent  with  this  data  collection  and  based  on  frequency  data,  the 
staff  selected  eight  common  problems  for  which  criteria  were  developed  to 
assess  accessibility  and  outcome.    Process  of  care  was  examined  through  re- 
view of  charts  of  those  patients  with  one  of  the  identified  problems.  This 
review  was  exploratory  and  was  designed  principally  to  begin  defining  the 
utility  of  process  or  outcome  reviews  to  assessing  quality  of  ambulatory 
care.    The  sample  distribution  of  the  first  questionnaire  was  740  well -person 
visits,  and  1560  problem-related  visits. 

Follow-up  questionnaires  were  returned  by  74%  of  the  1560  problem  visits, 
and  the  eight  problem  types  accounted  for  21%  of  the  problem  visits.  The 
analysis  conducted  for  the  eight  problems  involved  a  chart  review  of  patients 
having  one  of  the  problems  (327  cases  =  0.21  x  1560).    The  eight  common 
problems  included  otitis  media,  upper  respiratory  infection  (URI)  and  sore 
throats  (Pediatrics);  URI,  sore  throats,  hypertension  and  abdominal  pain 
(Adult  Medicine);  and  vaginitis  (0B-GYN);  and  were  examined  to  evaluate 
accessibility  and  problem  status  outcome. 

Three  sets  of  criteria  were  developed  for  this  study: 

1.  To  assess  accessibility  for  eight  problems 

2.  For  outcome  of  care 

3.  For  process  of  care 
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Criteria  for  accessibility  were  developed  by  a  panel  of  consumers,  admin- 
istrators and  physicians,  outcome  and  process  criteria  by  three  panels  of  CMP 
physicians  from  various  specialties.    Accessibility  criteria  were  set  for  two 
variables : 

1.  Access  interval,  defined  as  the  length  of  time  measured  in  days  be- 
tween the  first  contact  (to  arrange  visit)  and  the  visit 

2.  Contacts,  defined  as  the  number  of  interactions  an  enrollee  has  with 
clinic  personnel,  either  by  telephone  or  in  person,  to  arrange  a 
visit 

The  criteria  development  panel  established  acceptable  and  unacceptable 
values  for  each  problem  type.    Aggregate  results  were  presented  as  percentage 
distribution  of  patients  meeting  the  criteria.    These  data  were  taken  from  com- 
pleted questionnaires. 

The  criteria  for  effectiveness  of  care  (measuring  process  and  outcome)  were 
developed  by  CMP  physicians  through  group  discussions  and  consensus  within 
specialty  areas.    The  three  levels  of  review  in  Round  I  were: 

1.  Determining  compliance  with  explicit  process  criteria 

2.  Determining  compliance  with  explicit  outcome  criteria 

3.  Implicit  evaluation  of  medical  records  of  patients  not  meeting  all 
outcome  criteria  to  determine  reasons  for  poor  outcome 

Results  of  the  process  criteria  review  were  displayed  by  noting  the  per- 
centage of  rates  of  compliance  by  patients  presenting  one  of  the  eight  common 
problems.  The  CMP-EMCRO  staff  explored  the  relationship  between  the  explicit 
process  and  outcome  reviews,  reporting  that  although  little  statistical  rela- 
tionship existed  between  the  two  variables,  outcome  was  felt  to  be  improvable 
for  many  of  the  cases  with  substandard  outcome. 

Round  I  studies  were  concluded  with  a  presentation  of  study  results  at 
a  series  of  meetings  with  CMP  physicians  and  the  EMCRO  committee  aimed  at 
enabling  CMP  personnel  to  incorporate  results  into  program  management.    Round  I 
study  results  represent  only  the  first  step  in  total  program  development; 
CMP-EMCRO  personnel  noted  that  the  real  thrust  of  Round  I  was  to  begin  identi- 
fying methodological  issues  and  questions  to  be  examined  in  succeeding  studies. 


Round  II 

Round  II  was  designed  to  further  develop  and  test  the  outcome  methodology 
initially  implemented  in  Round  I.    Round  II  studies  were  based  on  a  sample  of 
CMP  enrollees  who  visited  the  clinic  between  May  22  and  June  17,  1974.  Two 
reports  were  initially  produced  from  this  sample: 

1.  "Accessibility  to  Visits  at  CMP,"  November,  1974 

2.  "Use  of  Patient-Reported  Status,"  May,  1975 
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Whereas  Round  I  results  were  compiled  and  reported  in  one  report,  the 
CMP-EMCRO  staff  reported  issues  and  results  separately  for  Round  II.  Most 
of  the  methodology  employed  in  Round  I  was  used  in  Round  II,  with  some  modi- 
fications.   The  number  of  common  problem  types  to  evaluate  was  expanded  to  seven- 
teen, adding  noninfective  dermatitis  in  both  Pediatrics  and  Adult  Medicine, 
conjunctivitis  and  allergic  rhinitis  in  Pediatrics,  back  pain  in  Adult 
Medicine  and  lacerations  and  removal  of  skin  growth  in  surgery.  Problems 
were  chosen  on  the  basis  of  frequency  of  occurrence. 

The  Accessibility  Panel  was  reconstituted  and  guidelines  from  Round  I 
were  reconsidered.  New  guidelines  were  developed  by  physicians  meeting  by 
specialty  groups,  who  sent  revisions  to  the  Panel  for  comment. 

As  a  result  of  the  Panel's  consensus,  criteria  were  specified  for  a 
maximum  number  of  days  a  patient  with  one  of  the  study  problems  should  have 
to  wait  between  initial  contact  with  the  clinic  and  time  of  visit.    The  panel 
decided  the  number  of  days  for  each  problem  by  first  or  follow-up  visit 
categories.    They  also  determined  that  the  maximum  days  allowed  should  be 
based  on  four  levels  of  symptom  intensity: 

1.  No  discomfort  or  pain 

2.  Some  discomfort  or  pain 

3.  Moderate  discomfort  or  pain 

4.  Considerable  or  extreme  discomfort  or  pain 

Information  measuring  symptom  intensity  was  recorded  as  part  of  the 
questionnaire  administered  during  the  patient  visit. 

Follow-up  visits  were  also  evaluated  for  accessibility.  Physician-ini- 
tiated visits  were  reviewed  by  CMP-EMCRO  staff  case  by  case.    The  medical 
record  was  reviewed  for  two  data  elements: 

1.  The  date  at  which  the  physician  told  the  patient  to  return 

2.  The  actual  date  the  patient  returned 

The  May  to  June,  1974,  questionnaire  was  administered  to  2,237  patients 
during  the  study  period,  of  which  1,719  were  problem-related  visits.  The 
17  problem  types  accounted  for  713  of  the  total  problem  visits.  Follow-up 
questionnaires  to  measure  outcome  were  mailed  to  all  patients  who  visited  for 
a  problem. 

The  Round  II  study  focused  on  the  CMP-EMCRO  goals  to  develop  and  test 
outcome  measures.    CMP-EMCRO  staff  indicated  problem  status  was  implicit  in 
the  method,  as  it  was  in  Round  I,  because  a  primary  purpose  of  ambulatory  care 
is  to  assist  in  the  resolution  of  problems;  thus  a  quality  assurance  method 
must  be  sensitive  to  this  important  objective.    Further  broader  health  status 
measures  (morbidity,  immunizations,  etc.)  were  too  insensitive  to  measure 
specific  problems  and  their  resolution.    The  questionnaire  measured  problem 
status  according  to  four  parameters: 

1 .  Symptom  frequency 

2.  Symptom  intensity 

3.  Degree  of  activity  limitation 

4.  Amount  of  patient  anxiety 
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In  terms  of  criteria  for  the  four  dimensions,  CMP  physicians  in  each 
specialty  set  explicit  outcome  criteria  for  the  selected  conditions  by  group 
consensus.    Physicians  determined  qualitative  maximum  levels  for  each  para- 
meter based  on  a  one-month  time  interval.    These  criteria  were  compared  with 
information  collected  from  a  second  mailed  follow-up  questionnaire  to  measure 
outcome  one  month  after  the  initial  visit.    The  same  four  parameters  were 
measured  in  the  second  questionnaire.    Both  questionnaires  required  the  patient 
to  rate  himself  on  a  scale  from  "none"  to  "extreme"  for  each  of  the  four  para- 
meters. 

Specific  study  components  of  Round  II,  as  a  result  of  the  cyclical  study 
design,  were  much  more  defined  in  the  manner  of  measurement  than  the  same 
Round  I  components.    In  addition  to  the  general  sample,  care  for  eleven  prob- 
lems was  extensively  evaluated.    This  analysis  generally  followed  Round  I 
methodology. 

1.  Problem-status  outcome  was  measured  and  criteria  applied 

2.  Medical  records  of  all  patients  failing  to  meet  outcome  criteria 
and  a  sample  of  those  meeting  criteria  were  reviewed  using  explicit 
process  criteria  as  well  as  implicit  judgments  to  determine: 

a.  Was  poor  outcome  due  to  error  in  care  process? 

b.  Could  this  level  of  outcome  be  explained  by  another  medical 
problem  which  complicated  this  episode? 

c.  Could  outcome  have  been  affected  by  progression  of  the  initial 
problem  into  another  illness? 

d.  Is  follow-up  problem  status  being  affected  by  another  episode 

of  the  illness  rather  than  poor  outcome  of  the  original  episode? 

The  major  component  of  Round  II  was  CMP-EMCRO  exploration  of  the  impor- 
tance of  cases  that  did  not  meet  outcome  criteria  (bad  outcome)  by  thoroughly 
examining  the  process  of  care.    This  was  accomplished  by  conducting  a  detailed 
medical  record  review  of  a  sample  of  cases  meeting  the  criteria  and  all  cases 
failing  the  criteria.    The  records  were  reviewed  using  both  implicit  and  explicit 
medical  criteria  for  a  number  of  selected  audits  (noted  above).    The  CMP-EMCRO 
staff  wanted  to  pinpoint  possible  differences  which  existed  in  the  medical  care 
process  for  cases  that  met  or  failed  the  outcome  criteria,  thus  relating  process 
to  a  measured  outcome.    The  differences  were  usually  associated  with  errors 
in  the  process  of  care.    This  type  of  review  enabled  the  CMP-EMCRO  staff  not 
only  to  link  process  with  good  and  bad  outcome,  but  also  to  report  independently 
the  results  of  a  process  review  of  selected  cases. 

Three  basic  judgments  were  made  during  the  review: 

1.  Judgment  of  diagnostic  process  based  on  the  explicit  criteria  (medi- 
cal record  elements  like  history,  physical  examination,  laboratory 
tests  used  to  establish  diagnosis) 

2.  Judgment  of  therapeutic  process  (correctness  of  treatment  regimen, 
follow-up,  other  instructions,  drugs  prescribed) 

3.  Judgment  of  diagnostic  outcome  (whether  diagnosis  was  correct  given 
the  information  in  medical  record  and,  if  not,  whether  the  correct 
diagnosis  should  have  been  made  sooner) 
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Judgments  were  made  by  physician  reviewers  after  explicit  review  had  been 
completed  by  CMP-EMCRO  staff.    The  object  of  this  three-level  review  was  to 
determine  reasons  for  poor  outcome.    Poor  outcome  could  be  related  to  deficien- 
cies in  the  process  of  care,  methodological  errors  in  problem  classification, 
errors  in  measurement  or  problem  status,  or  errors  in  explicit  criteria. 

An  important  part  of  the  methodology  and  review  process  is  the  use  of  an  > 
implicit  physician  review  procedure.    This  review  method  was  tested  for  metho- 
dologic  purposes  (i.e.,  to  test  the  utility  of  an  outcome-based  approach). 
It  is  not  being  advocated  as  a  practical  approach  to  continuing  evaluation  of 
care.    Total  care  must  consider  complicating  factors  (e.g.,  patient  characteris- 
tics, other  illnesses  and  social  problems)  to  accurately  evaluate  the  outcome 
of  specific  cases.    After  completing  the  review  of  the  cases,  the  physician 
reviewers  were  asked  to  classify  the  outcome  itself  in  one  of  three  ways: 

1.  Unimprovable  outcome  -  no  other  interventions  were  likely  to  improve 
outcome  for  a  specific  patient 

2.  Definitely  improvable  outcome  -  some  intervention  could  have  improved 
patient  outcome 

3.  Possibly  improvable  outcome  -  outcome  could  have  been  better,  but 
there  was  no  evidence  in  medical  record  that  would  definitely  achieve 
such 

The  base  information  used  in  this  evaluation  was  the  patient's  reported 
outcome  (problem  status)  gathered  from  the  follow-up  questionnaire.    These  re- 
views were  conducted  by  one  CMP  physician  who  evaluated  all  cases  and  a  second 
non-CMP  physician  who  reviewed  a  30%  sample  of  cases  to  insure  accuracy.  Both 
physicians  met  after  reviews  were  completed  to  resolve  disagreements,  and  to 
record  a  final  judgment. 


Telephone  Care 

The  study  of  telephone  calls  involved  assessment  of  the  content  and  quality 
of  telephone  care  "within  a  pediatric  practice  which  has  trained  health  assis- 
tants to  evaluate,  perform  triage  and  offer  home  management  advice  for  problems 
presented  by  parents."    Because  of  the  high  volume  of  pediatric  telephone  calls 
received  by  the  health  center  and  the  consequent  health  care  advice  provided 
over  the  telephone,  the  study  was  conducted  to  test  the  applicability  of  an 
outcome  strategy  for  assessing  the  quality  of  telephone  care.    The  purpose  of 
data  collection  and  evaluation  was  "assessment  of  end  results  of  care  for  prob- 
lems advised  by  telephone  in  lieu  of  a  clinic  visit."    CMP-EMCRO  staff  developed 
an  encounter  form  to  collect  the  following  data  elements  for  each  call: 

1.  Patient's  name  and  age 

2.  Purpose  of  the  telephone  communication 

3.  Nature  of  presenting  problem  (problem  status) 

4.  Disposition  of  the  case 

5.  Whether  or  not  consultation  was  obtained 
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Data  were  collected  from  2,520  telephone  calls  received  during  health 
center  hours  (October  14  to  November  9,  1974)  and  516  calls  during  weekends 
and  evenings.    One  week  after  initial  telephone  care  was  received,  a  follow- 
up  telephone  interview  was  conducted  by  an  independent  interviewing  agency 
in  264  of  those  cases  given  advice  in  lieu  of  a  visit.    The  resulting  infor- 
mation was  compiled  and  compared  with  established  criteria  developed  by  CMP 
pediatricians  in  a  departmental  meeting,  indicating  symptoms  and  potential 
disability  that  were  acceptable  outcomes  for  specified  problems.    There  were 
two  basic  problem  types:    immediate  care  and  administrative  (future  appoint- 
ments, assistance,  etc.). 

Access  to  telephone  care  was  measured  by  tabulating  the  number  of  attempts 
a  caller  made  to  obtain  care.    Caller  satisfaction  was  determined  by  asking 
parents  to  report  satisfaction  received  in  five  areas: 

1.  The  decision  to  care  for  the  child  at  home  rather  than  at  the  clinic 

2.  The  quality  of  home  management  advice  they  received 

3.  The  extent  to  which  their  questions  were  understood 

4.  The  extent  to  which  their  questions  were  answered 

5.  The  degree  of  interest  shown  by  the  provider 

Further  satisfaction  analysis  by  the  staff  resulted  in  development  of  a 
four-point  index  of  degree  of  satisfaction,  ranging  from  very  satisfied  to  very 
dissatisfied. 

To  assess  outcome,  the  staff  examined  cases  in  which  visits  occurred  after 
the  telephone  care.    A  clinical  evaluation  was  used  in  conjunction  with  the 
patient's  reported  problem  status  one  week  after  the  telephone  care  incident. 
This  assessment  was  designed  to  determine  two  points: 

1.  Whether  delays  in  receiving  treatment  after  initial  triage  decision 
were  evident 

2.  Whether  there  were  cases  of  unresolved  problems  in  which  patients 
did  not  secure  further  care 

A  chart  review  was  conducted  after  the  clinical  evaluation  was  completed  to 
determine  the  number  of  patients  diagnosed  with  treatable  and  nontreatable 
illnesses.    Chart  reviews  were  completed  on  14  cases  (0.05  x  247),  where  patients 
were  diagnosed  with  treatable  illnesses. 


Implementation  of  CMP-EMCRO  Results 

Although  quality  assurance  studies  are  not  formally  connected  with  the 
administrative  operations  of  CMP,  study  results  are  available  to  all  CMP  per- 
sonnel for  use  in  administrative  and  clinical  areas. 

Interaction  between  the  CMP-EMCRO  research  function  and  CMP  administra- 
tive activities  is  described  below, based  on  interviews  with  personnel  regarding 
this  specific  point,  and  two  memoranda  from  CMP-EMCRO  staff  to  CMP  clinical 
chiefs  regarding  process  reviews  for  urinary  tract  infection  and  sore  throat, 
and  review  of  care  for  hypertension. 
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In  terms  of  implementing  study  results  at  CMP,  several  areas  were 
identified: 

1.  The  accessibility  study  supported  administrative  actions  to  change 
appointment-making  procedures  by  identifying  several  key  problem 
areas  and  suggesting  alternative  administrative  solutions. 

2.  The  review  of  outcome  by  measuring  problem  status  for  vaginitis 
(patient-stated  problem  status)  indicated  physicians  in  the  OB-GYN 
department  were  not  communicating  with  patients  satisfactorily,  and 
that  patient  expectations  were  inflated.    This  subject  was  discussed 
with  physicians,  and  one  indicated  it  was  helpful  in  subsequently 
dealing  with  patients. 

3.  The  telephone  study  had  two  major  effects:    (a)  it  supported  the 
role  and  use  of  nonprofessional  health  assistants  in  providing 
pediatric  services,  and  (b)  it  changed  some  policies  for  handling 
telephone  care. 

4.  The  two  rounds  of  patient-reported  problem  status  outcome  evalua- 
tion and  subsequent  process  reviews  to  determine  reasons  for  poor 
outcome  led  in  one  department  to  educational  sessions  on  the 
management  of  urinary  tract  infection  and  hypertension. 

5.  Process  chart  reviews  have  been  analyzed  and  reported  to  the 
various  clinical  departments.    These  have  also  been  used  by  some 
physicians,  but  only  at  their  own  discretion. 

6.  CMP-EMCRO  project  personnel  were  asked  to  serve  on  an  ad  hoc 
committee  reorganizing  the  Adult  Medical  Department. 

Several  CMP  practitioners  who  have  particular  research  interests  or  want 
data  addressing  a  specific  problem  did  examine  EMCRO  data,  although  no  CMP 
physicians  except  the  principal  investigator  and  the  Chairman  of  Pediatrics 
participated  in  conceptualization  or  investigation  of  EMCRO  quality  assurance 
issues.    The  two  memoranda  document  interactions  regarding  specific  disease 
conditions  between  CMP-EMCRO  staff  and  CMP  clinical  chiefs.    The  memoranda 
refer  to  meetings  in  which  specific  EMCRO  reviews  were  discussed  and  outlines 
of  specific  EMCRO  suggestions  (e.g.,  medical  record  recording,  necessary 
medical  procedures  missing,  reduction  of  laboratory  tests,  lack  of  sufficient 
justification  for  diagnoses,  communicating  with  patients,  physicians'  educational 
sessions  and  guidelines  for  follow-up)  were  presented. 


ADDITIONAL  COMMENTS 

The  CMP-EMCRO  study  enjoys  considerable  support  in  four  important  areas: 

1.  The  patient  population,  predominantly  white  middle-class,  is 
receptive  and  responsive  to  patient  questionnaires  which  are  the 
data  base  for  EMCRO  studies. 

2.  The  medical  record  room  staff  assists  CMP-EMCRO  in  pulling  and 
processing  medical  records. 

3.  The  administrator  of  the  plan  is  supportive  of  research  activities. 

4.  The  medical  staff  willingly  assists  CMP-EMCRO  in  different  studies. 
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Round  III  data  collection  was  completed  between  October  and  November,  1975, 
but  results  were  not  available  at  the  time  of  this  report.    Round  III  sought  to 
develop  further  measures  of  access  and  outcome  and  to  test  their  utility  for 
quality  assurance  purposes.    This  round  also  included  a  random  sample  study 
(January,  1976)  of  the  CMP  membership  to  evaluate  access  to  care  in  the  popula- 
tion. 
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SECTION  III 

DELIVERY     SYSTEM  CHARACTERISTICS 


The  two  main  organizations  at  CMP  are  the  Connecticut  General  Life 

Insurance  Company  (CG)  and  the  Patuxent  Medical  Group,  a  physician  service 
corporation. 

Support  from  Connecticut  General  has  been  substantial  for  two  reasons: 

1.  Connecticut  General's  substantial  investment  in  the  community 
of  Columbia 

2.  Connecticut  General's  willingness  to  underwrite  the  delivery  of 
medical  services  through  alternative  funding  mechanisms 

The  Columbia  Medical  Plan  is  jointly  administered  by  Connecticut  General 
Medical  Programs  (CGMP)  and  the  Patuxent  Medical  Group;  CMP  operates  as  a 
division  of  the  CGMP. 


Administration  of  CMP  includes  the  following  specific  components: 

1.  The  Medical  Director  (from  physicians'  group)  and  Administrator 
(representative  of  CGO,  who  jointly  directs  the  operations  of 
CMP 

2.  Clinical  staff 

a.  4  FTE*  -  Internal  Medicine 

b.  3  FTE    -  Pediatrics 

c.  2.7  FTE  -  OB-GYN 

d.  1.8  FTE  -  Surgery 

e.  1 .8  FTE  -  Psychiatry 

f.  0.8  FTE  -  ENT/Ophthalmology  (contract  with  two  ophthalmologists 
for  0.6  FTE) 

g.  3  FTE  -  Medical  health  counselors 

h.  Orthopedic,  dermatologic  and  neurologic  services  are  contracted 

3.  Administrative  staff 

a.  1  controller 

b.  Administrative  assistant 

c.  Marketing  manager 

d.  Patient  relations  officer 

e.  36.0  FTE  receptionists,  secretaries,  telephone  operators, 
record  room  clerks 

f.  8.9  FTE  paramedic-health  assistants 

Services  offered  include  all  primary  services  in  pediatrics,  internal  medicine, 
OB-GYN,  ENT-ophthalmology ,  psychiatry  and  dermatology,  as  well  as  radiology, 
laboratory,  emergency  care  and  pharmaceutical  services. 

CMP  serves  a  population  of  20,000,  with  the  majority  from  Columbia, 
Maryland,  and  the  outlying  county  area.    Methods  of  payment  include  prepaid 

monthly  rates,  with  different  rates  for  individual,  family  and  employer-employee 
groups. 


*Full-time  equivalent 
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CMP 1 s  main  affiliation  is  with  Johns  Hopkins  University.  Specifically, 
CMP  is  affiliated  with  two  entities  within  the  Johns  Hopkins  Medical  Institu- 
tions, although  not  in  the  same  way:    the  Johns  Hopkins  Health  Services  Research 
and  Development  Center  and  the  Johns  Hopkins  School  of  Medicine. 

All  CMP  physicians  have  faculty  appointments  at  the  Johns  Hopkins  School  of 
Medicine.    CMP's  Research  Protocol  Review  Committee  continues  the  long-standing 
relationship  between  CMP  and  the  Johns  Hopkins  Research  and  Development  Center 
and  a  representative  of  the  Center  is  an  ex  officio  member  of  this  committee. 
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SECTION  IV 


H  C  MS    STUDY  INFORMATION 


CMP-EMCRO  agreed  to  participate  in  the  study  during  a  telephone  conversa- 
tion (December  9,  1975)  and  in  correspondence  (December  22,  1975).    They  were 
included  in  the  study  to  represent  a  site  where  quality  assurance  research 
activities  were  being  conducted  in  conjunction  with  the  implementation  of 
administrative  quality  assurance  activities.    The  interrelationship  between 
research  and  an  operational  delivery  system  is  important  in  determining  the 
status  of  CMP's  ambulatory  quality  assurance.    Initial  documentation  was  re- 
ceived from  CMP-EMCRO  in  December,  1975.    The  documentation  outlined  several 
research  studies  that  had  been  implemented  at  CMP  and  described  the  results 
of  the  studies.    The  documents  received,  prior  and  subsequent  to  the  site 
visit,  include: 

1.  An  experimental  Medical  Care  Review  Organization  in  an  HMO 
(October  1974) 

2.  The  Use  of  Patient-Reported  Problem  Status  in  Evaluating 
Ambulatory  Care  (May  3,  1975) 

3.  Quality  of  Telephone  Care:    Assessment  of  a  System  Utilizing 
Nonphysician  Personnel 

4.  Quality  Assurance  in  Ambulatory  Care:    A  Strategy  Based  on 
Outcome  Assessment  (April  6,  1976) 

5.  Quality  Assurance  in  Hypertension;  Memorandum  (July  31,  1975) 

6.  Process  reviews  for  urinary  tract  infection  and  sore  throat; 
Memorandum  (August  14,  1975) 

7.  Accessibility  of  Medical  Care  in  a  Health  Maintenance  Organization 
(March  20,  1974) 

8.  Accessibility  to  Visits  at  the  Columbia  Medical  Plan;  Second 
Round  Report  (November,  1974) 

9.  Evaluating  Access  to  Ambulatory  Care  (May  4,  1975) 

10.  Chart  Review  Form 

11.  Quality  Assurance  Program  Questionnaire  -  Adult  Medicine 

12.  Quality  Assurance  Program  Questionnaire  -  Pediatrics 

13.  Quality  Assurance  Follow-Up  Questionnaire  -  Pediatrics 

These  documents  were  thoroughly  reviewed  prior  to  the  site  visits  made  on  two 
occasions:    February  2,  1976,  and  April  23,  1976.    The  following  personnel 
were  interviewed: 

1.  Research  Project  Manager,  Columbia  Division  (EMCRO  Project) 

2.  EMCRO  Research  Associate  (responsible  for  data  collection, 
compilation  and  analysis) 

3.  Medical  Director,  Columbia  Medical  Plan,  and  President,  Physicians 
Service  Corporation 

4.  Chairman,  Pediatrics  Department,  Columbia  Medical  Plan 

5.  Plan  Administrator,  Columbia  Medical  Plan 

6.  Consultant  to  Columbia  Medical  Plan  from  the  John  Hopkins  Health 
Research  and  Development  Center  on  management  information  system 

7.  Director,  CMP  Drug  Utilization  Study 
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Total  interview  time  amounted  to  approximately  10  hours.  The  interviews  were 
not  audio  recorded,  but  investigators  compiled  extensive  hand-written  notes. 

General  topics  covered  during  the  interviews  were: 

1.  Background  information  regarding  the  development  of  CMP-EMCRO 

2.  Relationship  between  CMP  and  John  Hopkins  Health  Research  and 
Development  research  interests 

3.  Development  of  each  of  the  quality  assurance  studies,  including 
accessibility,  problem  status,  telephone  care  and  process 
reviews  of  specific  disease  conditions 

4.  Development  of  new  research  questions  as  EMCRO  was  implemented 

5.  Implementation  of  study  results  in  Columbia  Medical  Plan 
operations 

6.  Role  of  Physicians  Service  Corporation  in  research  activities 

7.  Role  of  Connecticut  General  (past,  present,  future)  at  CMP,  and 
its  interest  in  EMCRO  research  activities 

8.  Financial  status  of  CMP 

9.  CMP  use  of  EMCRO  study  results 

10.  Importance  of  active  research  activities  at  CMP 

11.  Specific  methodological  elements  of  the  various  quality  assurance 
studies 

The  majority  of  the  interview  time  was  spent  with  the  first  two  officers, 
although  a  minimum  of  40  minutes  was  spent  with  each  interviewee. 
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GIVEN  HEALTH  CARE  CENTER 
Burlington,  Vermont 


SECTION  I 
INTRODUCTION 


Given  Health  Care  Center  (GHCC)  is  a  privately  owned  nonprofit  group 
practice  situated  within  the  University  of  Vermont  Medical  School.    It  serves 
a  patient  population  of  approximately  11,000  primarily  from  Chittenden  County, 
Vermont,  and  provides  services  in  Adult  and  Adolescent  Medicine.    The  staff 
of  GHCC  initially  came  from  the  University  because  of  its  interest  in  developing 
a  small  ambulatory  group  practice  within  a  university  setting. 

GHCC  was  formed  for  the  following  three  basic  reasons: 

1.  To  teach  third  and  fourth  year  medical  students  in  a  small  group 
practice  situation  via  a  project  grant  from  the  Office  of  the  Dean 
of  Medicine  at  the  University  of  Vermont. 

2.  To  allow  students  to  practice  in  an  ambulatory  setting  where  staff 
would  provide  role  models  for  private  practice.    The  students  would 
also  learn  about  audit  procedures,  quality  assessment  activities, 
and  peer  review  functions.    This  approach  would  be  unique  in  training 
new  physicians  to  accept  quality  assurance  programs  as  part  of  the 
provision  of  medical  care. 

3.  To  use  GHCC  as  a  testing  ground  for  new  ways  to  deliver  efficient 
medical  care  and  to  determine  cost  requirements  for  providing  these 
services  to  an  expanding  population  by  a  small  core  of  providers. 

The  GHCC  practice  is  supported  largely  by  fee-for-service  financing, 
although  there  are  200  to  250  medical  students  who  receive  care  on  a  prepaid 
basis.    Out-of-pocket  expenses  account  for  25%  of  total  income;  Blue  Cross- 
Blue  Shield,  Medicare  and  Medicaid  account  for  50%;  and  group  employment 
insurance,  25%.    GHCC  research  and  development  are  supported  by  grants  from 
Kellogg  Foundation  and  Commonwealth  Fund.    These  grants  were  awarded  to  develop 
treatment  protocols,  patient  flow,  etc.,  for  patients  with  chronic  illness, 
and  to  develop  new  teaching  approaches. 

A  five-member  physician  staff  and  two  house  practitioners  provide  all 
primary  clinical  services.    GHCC  was  included  in  the  study  because  of  its 
substantial  experience  with  Problem  Oriented  Medical  Record  (P0MR)  audit  pro- 
cedures. 
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SECTION  II 

QUALITY     ASSURANCE  PROGRAM 


The  discussion  of  GHCC's  Quality  Assurance  Program  will  be  divided  into 
two  major  sections: 

1.  The  elements  of  GHCC's  Quality  Assurance  Program  documented  in  GHCC's 
Principles  of  Practice 

2.  The  operational  status  of  program  based  on  information  collected 
during  the  on-site  investigation 


BACKGROUND  AND  FORMALLY  PLANNED  ACTIVITIES 

Documentation  provided  by  GHCC  indicated  the  POMR  is  the  cornerstone  of 
the  GHCC  type  of  delivery  system  and  audit  review  program.    All  medical  en- 
counter information  is  typed  in  the  medical  record  to  facilitate  different  types 
of  audits.    The  POMR  system  has  three  operational  components: 

1.  A  system  of  care  based  on  the  four  phases  of  medical  action:  data 
base,  problem  identification,  plans  for  addressing  problems,  and 
fol low-up 

2.  Use  of  POMR  for  organizing  patient  information 

3.  Use  of  audit  for  all  providers  of  care  with  a  structured  method  of 
feedback  to  assure  adherence  to  standards 

The  documentation  notes  specific  requirements  for  compilation  of  data 
base  information,  periodic  evaluation  of  patients  after  initial  evaluation, 
identification  of  and  documentation  of  problems  and  associated  plans,  and 
follow-up  to  original  plans. 

Audit  goals  and  principles  are  described  as  an  integral  part  of  use  of 
the  POMR,  with  specific  audit  questions  outlined  for  each  POMR  component.  The 
purpose  of  audits  at  GHCC  is  administrative  as  well  as  medical: 

1.  To  assure  the  patient  specified  standards  of  care  are  met 

2.  To  provide  continuing  education  for  all  providers 

3.  To  evaluate  and  correct  the  standards  of  a  health  care  system  through 
the  feedback  of  information  generated  by  an  audit  system 

Audit  procedures  as  outlined  in  the  Principles  of  Practice  are  general, 
and  do  not  necessarily  apply  to  a  specific  audit  program.    Desirable  concepts 
and  attitudes  toward  audit  procedures  are  outlined,  while  some  components  that 
should  be  included  in  an  "effective"  audit  system  are  identified: 

1.  A  clearly  stated  organizing  principle  (e.g.,  GHCC's  is  a  POMR)  must 
be  selected 

2.  Providers  must  agree  explicitly  on  the  content  and  application  of 
the  system 
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3.    A  standard  means  of  organizing  medical  information  must  be  used 
Subjects  and  objects  of  audit  and  review  were  identifed  as: 

1.  Patients  (auditing  their  own  POMR) 

2.  Physicians,  nurses  and  paramedical  assistant  (peer  review  of  POMR) 

3.  The  delivery  system  (clerical  review  of  POMR) 

Questions  that  should  be  asked  as  each  area  of  the  POMR  is  audited  are 
fol lows : 

1.  Audit  of  Each  POMR  Component: 

a.  Data  Base 

T)    Have  the  data  that  physicians  and  nurses  agreed  to  collect 
been  collected?    If  not,  explain  (in  writing). 

2)  Have  abnormal  findings  been  acted  upon?    If  not,  why? 

3)  Have  individuals  complied  with  practice  standards?    If  not, 
need  evidence  of  support  for  disagreement. 

b.  Problems  List  -  Audit  Questions 

T)    How  thorough  has  the  individual  been  in  listing  all  problems 
evidenced  by  the  data  base? 

2)    Has  the  problem  been  defined  at  a  level  of  synthesis  for 
which  evidence  from  the  data  base  is  available?    When  avail- 
able data  lead  to  various  interpretations,  it  is  the  auditor's 
function  to  review  and  make  sure  plans  are  logical  and  will 
carry  the  problem  to  a  higher  level  of  resolution. 

c.  Plans  -  Audit  Questions 

T)    Is  the  plan  for  more  information  complete  so  the  problem  for 
which  the  patient  is  potentially  at  risk  is  reversible  or 
treatable?    Has  it  been  recognized  and  appropriately  ruled 
out? 

2)  Have  the  variables  required  for  the  management  of  the  problem 
been  recognized? 

3)  Is  a  clear  goal  stated  for  treatment  and  is  treatment  likely 
to  achieve  this  goal? 

4)  Is  there  a  plan  for  patient  education  and  is  it  stated  clearly 

d.  Follow-up  -  Audit  Questions:    As  new  data  become  available,  is 
the  problem  appropriately  modified? 

2.  Audit  of  Health  Care  System 

a.  What  happens  to  the  people  who  are  being  given  care  (the  study  of 
outcomes)? 

b.  How  efficiently  are  resources  used? 

c.  How  effective  is  leadership  within  the  system  in  asking  these 
questions?    Does  the  design  of  the  system  allow  for  rapid  response 
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Documents  indicate  each  provider  should  devote  an  hour  per  week  to  audi- 
ting, with  group  members  dividing  themselves  into  audit  pairs.    The  cost  of 
auditing  will  be  incorporated  into  the  overhead  expense  of  the  practice. 

There  are  six  actions  which  can  be  taken  by  the  entire  provider  group 
based  on  audit  results: 

1.  Disagree  with  the  audit  results 

2.  Decide  to  review  the  problem  again  in  one  month 

3.  Decide  to  change  the  auditor 

4.  Agree  with  the  audit  and  request  physician  to  correct  situation 

5.  Request  outside  consultant  to  review  the  problem  (member  being  audited 
has  the  same  right) 

6.  Request  for  resignation  if  member  fails  to  improve  performance  within 
a  period  of  six  months 

Principles  of  Practice  includes  four  audit  forms  designed  to  guide  each 
audit  process.    The  forms  outline  systematic  data  elements  or  questions  which 
must  be  addressed  and  are  entitled: 

1.  Patient  Review  of  Medical  Record 

2.  Clerical  Audit 

3.  Procedural  Audit 

4.  Physician  Audit 

These  forms  are  not  described  in  terms  of  their  use  in  specific  audits; 
they  seem  to  be  data  collection  forms  for  all  forms  of  review. 

A  final  area  of  review,  Data  Reduction,  is  outlined  as  follows: 

1.  Review  existing  records  for  problems  or  potential  problems  not  on 
problem  list 

2.  Organize  existing  data  into  POMR  format 

3.  Reduce  and  organize  existing  data  to  increase  physician  efficiency 

This  process  seems  to  be  incorporated  in  the  Clerical  Audit  described  in 
the  next  section,  Operational  Activities. 


OPERATIONAL  ACTIVITIES 

GHCC  personnel  interviewed  during  the  site  visit  reviewed  the  documented 
results  of  audits  described  in  this  section  with  investigators.  Investigators 
did  not  collect  any  written  documentation  (e.g.,  audit  results  or  minutes)  on 
the  audits. 

Five  distinct  audit  programs  were  identified  as  operational: 

1.  Patient  audit  of  own  records 

2.  Clerical  audit  of  POMR 
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3.  Procedural  audit  (data  reduction) 

4.  Quality  assurance  audit  (physician  and  problem-identified  audits) 

5.  Follow-up  audits 

The  Clerical,  Procedural  and  Problem-Identification  audits  (No.  2,  3  and  4) 
are  conducted  by  an  employed  auditor,  who  works  15  to  20  hours  per  week,  and 
is  also  available  for  special  audits.    Patient  audit  (No.  1)  is  performed  by 
the  patient;  follow-up  audits  (No.  5)  are  performed  by  physicians. 


Patient  Audit 

GHCC  staff  indicated  the  importance  of  consumer  involvement  in  all  aspects 
of  care.    The  patient  audit  procedure  begins  when  the  medical  record  is  sent 
to  each  new  patient  with  an  explanatory  cover  letter.    New  patients  are  those 
who  have  just  entered  the  GHCC  patient  population,  and  have  had  their  initial 
health  evaluation  (data  base)  completed.    The  patient  reviews  the  entire  record 
to  determine  if  information  is  correct  and  if  his  interpretation  of  the  prob- 
lem is  the  same  as  that  of  the  provider.    He  is  asked  to  look  at  the  priority 
statement  and  decide  what  problem  should  be  treated  first;  these  provider- 
compiled  priority  statements  are  stated  in  the  actual  document.    The  patient 
reviews  the  "SOAP"  component  of  the  POMR,  which  is  defined  as: 

"S"  (subjective  data)  -  Historical  information  gathered  from  patient. 
Patient  is  to  read  carefully  and  check  for  accuracy. 

"0"  (objective  data)  -  Results  of  tests  and  examinations  related  to  the 
problem. 

"A"  (assessment)  -  Interpretation  of  the  findings  from  "S"  and  "0"  if 
needed  to  clarify  problem. 

"P"  (plan)  -  The  recommended  program  of  treatment  and  what  providers  feel 
should  be  done  for  the  problem. 

Approximately  100  to  200  new  patients  receive  an  initial  evaluation  each 
month;  there  is  a  one-week  turn-around  between  the  health  examination  and  the 
patient's  study  of  the  record.    GHCC  personnel  indicated  approximately  8%  of 
records  were  returned  with  comments.    The  records  not  returned  are  considered 
to  accurately  reflect  a  patient's  problem  list,  and  GHCC  personnel  proceed 
on  that  basis  in  delivering  further  care. 

GHCC  personnel  indicated  an  important  goal  of  the  audit  is  to  affect 
utilization  by  involving  patients  in  their  own  care.    GHCC  feels  it  can  be 
inferred  from  self-audited  records  that  patients  know  more  about  and  agree  with 
their  problem  lists,  thereby  reducing  the  propensity  to  seek  unnecessary  care. 


Clerical  Audit 

The  clerical  audit  judges  the  use  of  the  POMR  by  GHCC  personnel  by  means  of 
two  sources:    (a)  the  initial  health  evaluations,  and  (b)  a  random  sample  of 
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charts.    The  auditor  checks  for  patient  and  provider  identification,  and  for 
the  following  items: 

1.  Data  base  collection  (both  history  and  physical) 

2.  Physician  findings 

3.  Commission  rates 

4.  Omission  rates 

5.  Initial  evaluation  of  the  patient 

6.  Plan  for  follow-up 

The  audit  sheet  stays  in  the  medical  record  along  with  the  generated  defi- 
ciency list  and  the  record  is  forwarded  to  the  provider  for  corrections.  A 
deficiency  usually  focuses  on  one  of  five  areas: 

1.  Problem  must  be  formulated 

2.  Patient  must  be  called  for  re-visit 

3.  Some  data  must  be  corrected 

4.  Data  is  recognized  (not  in  format),  no  further  action  necessary 

5.  Data  is  incorporated  in  problem  list,  no  further  collection  necessary 

During  the  clerical  audits  of  the  initial  health  evaluation,  the  extensive 
patient  questionnaire  (PROMIS  II)  is  reviewed  to  determine  whether  the  initial 
examination  produced  reliable  problem  identification,  plans  and  follow-up  plans. 
This  requires  the  auditor  to  spend  significant  amounts  of  time  with  each  chart 
to  determine  the  efficacy  of  SOAP,  as  well  as  to  verify  the  objective  medical 
information. 

GHCC  personnel  indicated  the  omission  rate  was  4%.    The  cost  of  the  audit 
was  placed  at  $.50  per  chart.    The  number  of  clerical  audits  coincides  with  the 
number  of  initial  health  evaluations,  but  may  include  some  periodic  health 
evaluations. 


Procedural  Audit 

This  audit  was  conducted  along  with  the  initial  implementation  of  the 
POMR.    During  the  first  two  years  of  POMR  use,  GHCC  personnel  stated  that  a 
strict  monitoring  system  was  instituted  to  insure  use  of  the  problem-oriented 
technique.    The  monitoring  audit  asked  14  yes  or  no  questions  about  the  content 
of  the  medical  record.    The  auditor  was  required  to  note  the  patient's  name, 
physician's  name,  auditor's  name,  date  of  audit  and  number  of  problems  listed 
in  the  record.    The  last  part  of  the  audit  was  a  "comments"  section  designed 
to  outline  corrective  action. 

GHCC  personnel  said  this  audit  was  not  being  conducted  "much  anymore," 
but  is  still  performed  randomly. 


Quality  Assurance  Audit  (Physician) 

The  physician  audit  procedure  requires  a  team  of  two  GHCC  physicians  to 
review  two  charts  per  week.    The  teams  stay  together  for  a  three-month  period, 


37 


and  examine  records  for  data  justifying  the  problem  statements,  the  treatment 
plans  outlined,  and  whether  the  POMR  goal  statement  is  based  on  data  in  the 
record.    This  goal  statement  is  the  basis  for  follow-up  audits  which  can  be 
conducted  at  some  other  point  in  a  patient's  history  at  GHCC. 

A  major  focus  of  physician  audits  is  to  examine  prescribed  medications 
to  insure  they  are  necessary,  based  on  the  record. 

Records  are  chosen  for  audit  at  random.    Physicians  perform  implicit  peer 
review  of  care,  guided  by  a  format  which  requires: 

1 .  Patient's  name 

2.  Name  of  provider  being  reviewed,  and  names  of  providers  conducting 
review 

3.  Date  of  audit 

4.  Audit  of  the  data  use,  problem  foundation,  plans  and  follow-up  for 
each  record 

5.  Suggestions  and  reasons  for  action  in  each  area  identified  in  No.  4 

6.  Source  of  information  within  chart  which  audit  decision  was  based  on 

Audit  results  were  reported  to  be  discussed  in  weekly  staff  meetings. 
Study  staff  reviewed  a  small  sample  of  medical  staff  meeting  minutes,  but  they 
contained  no  discussions  of  audit  results. 


Quality  Assurance  Audit  (Problem  Identification) 

The  problem  identification  audit  is  based  on  interaction  among  GHCC  provi- 
ders, who  choose  certain  diagnoses  for  review.    Four  conditions  have  been  chosen 
and  updated:    (a)  hypertension,  (b)  diabetes,  (c)  anemia,  and  (d)  hyperl ipidemia. 
Approximately  200  cases  of  each  of  these  four  conditions  have  been  received.  Cleri- 
cal personnel  (the  auditor)  conduct  the  reviews,  based  on  explicit  standards 
developed  by  the  provider  staff. 

This  audit  area  was  kept  as  flexible  as  possible.    Outside  consultants 
have  been  hired  to  conduct  problem  identification  audit,  but  not  as  a  normal 
practice.    GHCC  prefers  to  identify  problems  within  the  practice,  and  to  collect 
data  that  allow  judgment  regarding  quality  of  care.    Action  is  taken  after  audit 
results  have  been  discussed  at  weekly  provider  staff  meetings. 

Study  staff  collected  no  evidence  regarding  use  of  audit  results  within 

GHCC. 


Fol low-Up  Audits 

Follow-up  audits  are  conducted  on  all  patients  who  have  visited  GHCC  three 
or  more  times  in  one  year.    The  audit  is  geared  to  focus  on  documented  goal 
statements  within  the  records,  and  to  judge  whether  the  provider-patient 
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encounters  are  accomplishing  that  goal.  For  each  succeeding  visit,  the  auditor 
looks  for  more  specific  reasons. 


ADDITIONAL  COMMENTS 

Much  of  the  activity  at  GHCC  is  oriented  to  clerical  audit  of  the  initial 
health  evaluation.    The  total  program  is  characterized  by  flexibility  of 
various  audits  which  focus  on  a  variety  of  subjects. 

The  most  unusual  characteristic  of  GHCC  is  the  input  of  the  consumers  with 
regard  to  the  total  provision  of  care.    Presently,  results  are  being  used  only 
as  they  apply  to  the  patients  being  seen  at  GHCC.    It  is  the  future  goal  of 
GHCC  to  provide  a  mechanism  for  applying  results  to  patients  taken  care  of 
outside  this  practice. 

Study  staff  were  given  four  sets  of  minutes:    two  dealt  with  audit  activi- 
ties; the  other  two  dealt  with  general  health  system  issues.    Because  of  the 
fragmentation  in  the  minutes,  no  information  was  used  from  this  source  to 
substantiate  operational  activities. 


230-045  O  -  77  -  4 
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SECTION  III 


DELIVERY    SYSTEM  PROGRAM 


SERVICES 


A  full  range  of  ambulatory  care  is  provided  to  the  GHCC  population,  inclu- 
ding an  extensive  patient  education  program.    Many  letters  and  pamphlets  are 
mailed  to  them  relating  to  sore  throat,  contraceptives,  medicines,  urinary 
tract  infections,  viral  infections,  mononucleosis,  beta  hemolytic  streptococcus, 
chronic  prostatitis,  poisoning  and  dental  care. 

In  addition  to  regular  outpatient  department  hours,  GHCC  is  open  at  nights 
and  on  weekends;  staff  members  rotate  shifts. 


PATIENT  POPULATION 

In  1975,  GHCC  treated  8,000  patients  who  made  14,600  office  visits  between 
January  1  and  December  31.    The  population  is  99%  white,  and  lives  mainly  in 
Burlington,  Vermont,  although  some  patients  come  from  as  far  as  New  Hampshire. 
Some  patients  are  from  the  private  practices  of  the  staff  physicians.  The 
population  is  sophisticated,  with  a  high  educational  and  income  level.  The 
no-show  rate  is  very  low  and  patient  audits  usually  show  a  reasonable  response 
rate. 


The  election  of  new  employees  or  members  of  GHCC  staff  to  the  status  of 
full  members  is  dependent  on  the  levels  of  thoroughness,  reliability,  synthe- 
sizing ability  and  efficiency  they  display. 

The  distribution  of  FTE  staff  by  profession  at  GHCC  is  as  follows: 


STAFF 


PROFESSION 


FTE 


Physicians 

Nurse  Practitioners 

Administrative  Personnel 

Paramedics 

Transcribers 

Lab  Technician 

Clerk  Auditor 

Medical  Records  Personnel 

Secretaries 

Switchboard  Operator 


4 

1-2/5 


2 
3 


2 
5 


2-1/2  (1/2  university  related) 
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Staffing  costs  amount  to  approximately  $246,000.    Of  this,  the  medical  staff 
accounts  for  $97,000  (MD  -  81,000;  RN  -  21,000);  and  the  Administrative  Staff  - 
$144,000. 


MEDICAL  RECORDS  SYSTEM 

The  medical  record  is  the  POMR;  the  POMR  format  provides  data  elements 
for  a  controllable,  uniform  data  base  collected  when  a  patient  first  comes 
to  GHCC  and  receives  a  physican  examination.    A  periodic  evaluation  (normal 
routine  check-up)  redefines  the  data  base  after  the  initial  complete  evalua- 
tion.   If  a  patient  comes  in  for  required  visits,  a  plan  is  put  into  the  POMR 
which  defines  any  additional  data  base  needed  for  periodic  evaluation.  The 
frequency  of  periodic  evaluations  and  changes  in  the  data  base  are  defined  by 
the  practice  group.    To  indicate  the  time  is  appropriate,  the  paramedic  will 
write  "Health  Maintenance  Evaluation"  in  the  record. 

The  objectives  of  the  Health  Maintenance  Evaluation  are  as  follows: 

1.  To  screen  on  a  regular  basis  for  problems  that  are  important 

2.  To  update  the  patient's  medical  record 

3.  To  provide  health  education  for  the  reinforcement  of  good  health 
habits  and  to  change  poor  health  habits 

Rules  have  been  established  for  data  reduction  (what  to  do  in  the  system 
if  there  is  an  error)  in  the  POMR  process.    They  cover  topics  such  as  what 
to  do  with  letters  for  other  physicians  and  how  to  record  specific  information 
on  the  data  sheet. 


Problem  Lists 

Physicians  are  responsible  for  insuring  that  the  patient  has  an  updated 
list  of  all  medical,  behavioral  and  social  problems.    Each  active  problem 
must  have  a  clearly  detailed  plan  for  evaluation,  treatment  and  patient  educa- 
tion.   A  complete  problem  list  provides  an  explicit  data  base  to  be  used  as  a 
standard  for  evaluation  of  problem  statements. 

When  there  is  no  problem  list  for  the  patient,  the  data  base  is  incomplete. 
A  complete  POMR  data  base  requires  a  problem  list  for  any  patient  regularly 
receiving  care. 

Each  active  problem  will  have  a  written  plan  as  follows: 

1.  Use  the  rule-out  process  which  links  acquired  data  to  a  specific 
problem  which  is  being  ruled  out. 

2.  Include  parameters  which  are  required  for  management. 

3.  State  treatment  clearly  enough  for  patient  or  any  physician  to 
understand.    Expected  goal  should  be  stated. 

4.  In  the  patient  education  section  list  information  patient  must  have 
to  understand  the  problem  and  treatment.    The  patient's  area  of 
responsibility  must  be  defined. 
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SECTION  IV 

HCMS     STUDY  INFORMATION 


GHCC  agreed  to  participate  in  the  study  on  October  15,  1975,  via  correspon- 
dence, after  telephone  discussions  with  one  member  of  the  GHCC  physician  staff. 
Initial  documentation  was  received  on  February  2,  1976. 

The  documentation  received  and  compiled  prior  to  the  site  visit  on  March  4, 
1976,  is  as  follows: 

1.  The  Given  Health  Care  Center  Training  Manual,  approximately  250  pages. 
This  manual  describes  the  entire  practice  of  care  at  GHCC.    It  is  used 
to  educate  new  physicians  and  other  providers  with  regard  to  the  POMR, 
audit  forms,  chronic  disease  protocols,  patient  education,  chart  organi- 
zation and  rules  and  standards  for  practice.    This  document  is  also 
used  as  a  referral  source  by  physicians  and  other  providers. 

2.  Minutes  from  a  Special  GHCC  Business  Meeting  (December  13,  1975)  sum- 
marizing a  discussion  of  the  periodic  examination  of  the  data  base. 

3.  Minutes  from  the  Staff  and  Personnel  Meeting  (September  23,  1975). 

4.  Minutes  of  the  Second  Meeting  Regarding  Audit  (July  2,  1975) 

5.  Summary  Notes  of  the  Audit  Meeting  (June  9,  1975) 

6.  Cover  letter  for  record  that  is  sent  to  patient 

The  documentation  did  not  thoroughly  describe  the  type  of  Quality  Assurance 
audits  conducted  at  GHCC;  in  particular  more  information  on  the  POMR  is  needed. 
The  documentation  is  limited  in  that  only  one  set  of  audit  minutes  was  sent, 
making  accurate  description  of  the  specific  program  components  difficult. 

A  large  amount  of  information  was  collected  during  the  site  visits.  The 
interviews  were  not  audio  recorded.    Questions  were  asked  of  the  following  GHCC 
personnel : 

1 .  Staff  Physician 

2.  President  of  University  Associates  in  Comprehensive  Health  Care 

3.  Administrative  Assistant  (Office  Supervisor) 

4.  Paramedic 

5.  Clerical  Auditor 

Topics  discussed  during  the  interviews  included: 

1.  Development  of  the  Audit  Program 

2.  Goals  and  objectives  of  the  Audit  Program 

3.  Philosophy  and  responsibility  of  staff 

4.  Importance  of  consumer  involvement 

5.  Problem  Oriented  Medical  Record  Data  Base 

6.  Overview  of  the  four  audit  processes 

7.  Importance  of  audit  procedures  relative  to  total  delivery  system 

8.  Funding  of  Audit  Program 

9.  Cost  of  auditing  activities 

10.  Decision  making  in  Audit  Program 


42 


11.  Unique  characteristics  of  the  system 

12.  General  audit  and  quality  assurance  issues 

13.  Future  directions  of  GHCC  Audit  Program 


The  Staff  Physician  presented  an  overview  of  the  Audit  Program  as  well  as 
the  practice;  the  President  of  University  Associates  provided  staff  with  infor- 
mation regarding  the  critical  philosophy  of  the  physicians  as  well  as  the  types 
of  POMR  procedures;  the  Paramedic  presented  specific  information  on  the  audit  of 
the  data  base;  the  Administrative  Assistant  provided  information  on  staff,  re- 
sources, utilization,  and  financial  issues,  as  well  as  an  in-house  assessment 
of  the  program  in  1975;  and  the  Clerical  Auditor  provided  first-hand  knowledge 
of  and  experience  with  audits. 
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NORTH  CENTRAL  HEALTH  PROTECTION  PLAN 
Wausau,  Wisconsin 


SECTION  I 
INTRODUCTION 


The  Health  Care  System  Division  (HCSD)  of  Employers  Mutual  Liability 
Insurance  Company  of  Wausau,  Wisconsin,  manages  five  Individual  Practice 
Association  (IPA)  Health  Maintenance  Organizations  (HMO)  in  Wisconsin.  These 
organizations  are  termed  health  protection  plans  (HPP)  from  HCSD's  perspec- 
tive, and  contractual  arrangements  are  twofold: 

1.  HCSD  provides  complete  managerial  and  administrative  services  to  the 
IPAs. 

2.  HCSD  has  individual  contractual  agreements  with  all  individual  physi- 
cians participating  in  the  health  protection  plans. 

The  North  Central  Health  Protection  Plans'  (NCHPP)  quality  assurance 
activities  were  selected  for  review,  assessment  and  inclusion  in  this  study. 
The  NCHPP  is  located  in  Wausau,  Wisconsin,  and  includes  approximately  96  physi- 
cians.   Each  participating  physician  signs  an  agreement  with  HCSD,  which  includes 
four  components: 

1.  The  physician  will  agree  to  and  abide  by  the  decisions  of  the  Peer 
Review  and  Physicians  Operating  Committees. 

2.  The  physician  will  provide  a  fee  schedule  and  guarantee  those  fees 
for  a  12-month  period  (to  be  renegotiated  yearly). 

3.  The  physician  will  accept  the  risk  sharing  provisions  of  the  program 
(recently  raised  to  100%  sharing  of  the  Physicians  Fund). 

4.  The  physician  will  agree  to  and  abide  by  the  provisions  of  the  hospital 
concurrent  Review  Process  called  the  Hospital  Certification  Program. 

Within  Employers  of  Wausau,  the  HCSD  receives  management  direction  from 
the  Health  Care  Committee  (HCC),  consisting  of  five  senior  vice  presidents  of 
the  company.    The  division  is  directly  managed  by  an  assistant  vice  president 
and  division  manager.    A  health  protection  plan  administrator  works  directly 
with  the  respective  IPA,  his  functions  being  supervised  by  the  HCSD  manager. 

Each  IPA  is  operated  by  a  Physicians  Operating  Committee,  which  represents 
the  physician  members  of  the  local  medical  society  (for  NCHPP,  the  Marathon 
County  Medical  Society).    The  HCSD  deals  directly  with  the  Operating  Committee 
through  the  HPP  administrator,  as  well  as  with  the  Peer  Review  Committee,  a 
subcommittee  of  the  Operating  Committee.    Employers  of  Wausau' s  HCSD  partici- 
pation in  IPA  began  in  1972,  in  an  effort  to  positively  affect  the  cost  of 
physician  and  hospital  services  through  a  shared-risk  provision,  and  an  aggres- 
sive quality  and  utilization  monitoring  scheme.    The  main  services  performed 
by  HCSD  include  provider  relations,  underwriting  and  pricing,  marketing, 
actuarial,  policyholder  and  enrollee  relationships,  and  data  support. 
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The  IPA  includes  solo  practitioners,  small  group  practices  (two  to  ten 
physicians)  and  a  free-standing  medical  center.    The  study  examines  NCHPP's 
Peer  Review  activities  as  a  committee  encompassing  review  of  all  physicians 
within  the  plan,  and  does  not  focus  on  one  delivery  setting  or  review  situa- 
tion. 


45 


SECTION  II 

QUALITY       ASSURANCE  PROGRAM 


Because  of  the  similarity  between  documented  (planned)  activities  and 
operational  activities  of  NCHPP,  they  will  be  discussed  in  one  section  rather 
than  separately.    When  information  has  been  taken  from  documentation  or  the 
site  interview,  the  source  will  be  noted.    This  section  will  describe  NCHPP' s 
quality  assurance  program  as  follows: 

1.  General  outline  of  quality  assurance  system 

2.  Role  of  the  automated  data  collection  system  in  quality  assurance 

3.  Specific  activity  of  Peer  Review  Committee  and  its  members 

4.  HCSD  role  in  quality  assurance  system 

5.  Results  produced  by  quality  assurance  system  (peer  review  activities) 

6.  Results  of  study  staff's  review  of  Peer  Review  Committee  minutes. 

The  focus  of  ambulatory  quality  assurance  activities  at  NCHPP  is  a  peer 
review  process  conducted  by  a  nine-member  Peer  Review  Committee  (PRC),  made  up 
of  practicing  NCHPP  physicians.    Committee  activities  are  supported  by  HCSD 
staff  and  the  NCHPP  data  processing  capabilities. 

The  interaction  between  the  HCSD  (Health  Care  Systems  Division  of  Employers 
Insurance),  Peer  Review  Committee  (PRC)  and  Physicians  Operating  Committee 
(POC)  is  the  cornerstone  for  the  review  and  monitoring  system.    NCHPP  personnel 
indicated  that  because  all  NCHPP  physicians  are  sharing  the  risk,  they  are 
seriously  committed  to  an  operational  monitoring  system.    Interview  data  showed 
that  NCHPP  funding  comes  from  a  finite  fiscal  base,  so  utilization  must  be 
carefully  monitored. 

Individual  PRC  members  are  responsible  for  reviewing  a  certain  number  of 
family  or  individual  medical  profiles  (Account  Profiles).    Reviews  are  con- 
ducted monthly  and  written  reports  are  submitted  with  recommendations  to  the 
PRC.    The  committee  then  discusses  individual  review  cases,  and  takes  action 
relative  to  the  NCHPP  physician.    If  a  reviewed  physician  responds  to  committee 
action,  the  committee  will  then  review  and  discuss  his  reponse.    The  review 
orientation  requires  committee  physicians  to  relate  the  problem  or  diagnosis  to 
specific  procedures  noted  on  a  profile,  in  an  attempt  to  delineate  questionable 
utilization  of  resources  and  care. 


PROGRAM  ELEMENTS 

Encounter  claim  forms  are  the  data  source  used  for  all  informational 
requirements  for  NCHPP.    Each  specialty  or  individual  physician  joining  NCHPP 
develops  an  encounter  form,  in  consultation  with  HCSD  personnel.    Forms  are  not 
only  created  for  each  specialty,  but  also  for  individual  physicians  within  each 
specialty.    Encounter  claim  forms  are  either  sent  to  HCSD  for  automated  data 
processing  or  entered  directly  via  terminals  at  the  doctor's  office  in  an  on- 
line real-time  environment.    The  encounter  forms  are  precoded  and  include  the 
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diagnosis,  procedures  and  fee  schedule  for  the  provider.    The  forms  are  very 
explicit  and  itemize  in  extensive  detail  many  procedures  for  prospective 
billing. 

Two  types  of  coding  used  in  the  system  are  CPT  (for  treatment  and  pro- 
cedures) and  H-CDA  (diagnosis).    Generally,  physicians  use  the  1969  California 
Relative  Value  Scale  to  calculate  fee  schedules.    Accuracy  and  validity  checks 
of  the  coding  system  are  made  to  insure  a  low  error  rate  in  documents.  They 
are  as  follows: 

1.  Encounter  forms  are  compared  with  individual  profiles  to  check 
accuracy  of  diagnosis,  procedures  and  fees 

2.  The  Administrator  of  NCHPP  closely  supervises  input  data  clerks 
(telecommunicators )  who,  with  the  aid  of  the  system: 

a.  Verify  eligibility 

b.  Through  prepricing,  ensure  adherence  to  guaranteed 
fee  schedules 

3.  A  physician  specialty  comparison  is  periodically  produced  to 
identify  "high  utilizers."  Those  physicians  are  reviewed  by 
the  committee. 

4.  Physicians  code  and  check  forms  during  the  patient  encounter  to 
maximize  accuracy  and  validity.    This  system  of  checking  has 
been  used  for  two  and  a  half  years. 

Encounter  claim  data  are  the  basis  of  45  reports  produced  for  NCHPP  by 
Employers  Insurance  of  Wausau.    The  Account  Profile  report  used  for  all  ambu- 
latory reviews  by  committee  physicians  details  all  procedures,  problems, 
diagnoses  and  charges  submitted  by  NCHPP  physicians  for  each  memeber  of  a 
family  over  a  period  of  time  (approximately  one  to  two  years).    This  allows 
physicians  to  review  a  total  episode  of  care  for  an  entire  family. 

The  reviewing  physician  reviews  the  printout  of  the  Account  Profile, 
which  consists  of  several  data  elements: 


1. 

Account  number 

2. 

Data  printout 

3. 

Family  member  names,  sex  and  number 

4. 

Practice  code 

5. 

Date  of  service 

6. 

Procedure  code 

7. 

Brief  description  of  procedure 

8. 

Fee 

9. 

Diagnostic  codes  and  descriptions 

The  profile  reports  family  encounters  by  individual  member,  in  chronological 
sequence.    Since  these  profiles  are  randomly  selected  for  review,  a  reviewing 
physician  may  review  a  profile  with  a  few  encounters  or  with  a  number  of 
encounters,  thereby  making  the  review  workload  unpredictable.    The  NCHPP  Admin- 
istrator directs  a  clerk  to  acquire  a  copy  of  the  "Numeric  Report,"  which  lists 
all  social  security  numbers  of  NCHPP  members.    Ten  pages  are  chosen  at  random, 
then  a  column  from  each  page.    Ten  profiles  are  selected  from  the  report,  which 
provides  the  profiles  to  be  reviewed  that  month. 
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The  family  names  and  individual  physicians'  names  are  removed  to  preserve 
confidentiality  and  allow  anonymous  review.    The  Account  Profiles  are  referred 
for  review  to  Committee  members  by  the  Plan  Administrator.    Reviews  are  con- 
ducted individually  by  each  Peer  Review  Committee  physician.    The  two  committee 
physicians  interviewed  said  profiles  are  examined  to  detect  discrepancies  in 
total  episodes  of  care  and  errors  in  medical  logic.    The  physician  will  attempt 
to  correlate  particular  diagnoses  or  problems  with  noted  procedures  and  treat- 
ment regimens.    This  review  process  requires  each  reviewing  physician  to  make 
an  implicit  value  judgment  regarding  another  physician's  practice  in  specific 
cases . 

Once  a  committee  physician  has  reviewed  a  profile  and  made  some  judgments, 
he  is  required  to  submit  a  written  report  to  the  committee,  through  the  NCHPP 
Administrator.    The  written  report  requires  the  physician  to  outline  the  case, 
the  problems  or  issues  discovered  and  his  recommendations.    Once  the  committee 
meets,  each  case  is  discussed  and  a  full  committee  decision  is  reached  regarding 
specific  actions. 

The  committee  has  had  some  problems  in  getting  all  review  physicians  to 
write  reports  on  their  review  cases.    The  minutes  indicated  some  cases  are 
verbally  presented  to  the  committee.    However,  three  physicians  who  are  more 
involved  in  committee  activities  than  others  (serving  on  both  the  POC  and  the 
PRC)  provide  sufficient  enthusiasm  for  the  committee  to  function  as  an  active 
monitoring  body. 

Since  a  main  role  of  HCSD,  according  to  informants,  is  to  market  Health 
Protection  Plans  to  different  communities,  HCSD  was  extremely  careful  when  first 
establishing  the  PRC  to  obtain  physician  representatives  with  solid  reputations 
who  could  establish  committee  credibility  during  the  organization's  formative 
stages . 

As  noted  in  the  documentation,  the  PRC  is  comprised  of  nine  members;  three 
hold  joint  membership  on  the  POC,  and  six  are  elected  directly  to  the  PRC. 
Candidates  are  nominated  and  elected  by  members  of  the  County  Medical  Society. 
Both  committees  represent  physicians  who  belong  to  the  Marathon  Medical  Society, 
and  who  are  also  NCHPP  members.    Either  committee  may  refer  questions  or  pro- 
blems to  the  other  committee  for  comment  and  review,  although  the  POC  has  final 
responsibility  on  all  matters . 

As  noted  in  the  interview,  the  PRC  has  been  fairly  stable  since  only 
one-third  of  the  committee  turns  over  annually.    Committee  members  meet  on  the 
first  Tuesday  of  every  month.    Information  pertaining  to  review  is  received 
by  members  one  week  before  the  meeting,  except  for  account  profiles  which  are 
sent  at  least  two  weeks  before  the  meeting. 

The  documented  guidelines  for  the  Peer  Review  Committee  of  NCHPP  are  as 
follows : 

1.  The  Committee  will  be  known  as  the  NCHPP  Peer  Review  Committee. 

2.  The  Committee  shall  have  as  its  goal  the  improvement  of  medical 
care  in  the  community,  which  includes  inpatient  care,  hospital 
outpatient  care,  clinic  outpatient  care,  and  sound  medical  office 
practice. 
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3.  The  Committee  shall  have  a  membership  of  nine  persons  whose 
terms  shall  be  staggered  in  the  following  manner:    three  members 
with  three-year  terms,  three  members  with  two-year  terms,  and 
three  members  with  one-year  terms.    The  Committee  chairman  shall 
be  a  physician,  an  experienced  member  of  the  PRC,  and  a  nonvoting 
member  of  the  POC  (unless  a  member  of  the  Operating  Committee  by 
joint  election) . 

4.  Three  members  of  the  PRC--one  with  a  three-year  term,  one  with  a 
two-year  term,  and  one  with  a  one-year  term—shall  serve 
concurrently  on  the  POC. 

5.  The  Committee  shall  consist  of  physicians  from  the  various 
modes  of  practice,  including  family  medicine,  internal  medicine, 
ophthalmology,  other  various  specialties,  and  psychiatry. 

6.  Employers  Insurance  shall  be  represented  on  the  Committee  for 
interpreting  data,  and  a  nurse  coordinator  at  the  hospital  shall 
be  available  for  consultation  on  the  hospital  certification  pro- 
gram. 

7.  The  voting  members  of  the  Committee  shall  consist  of  physicians 
only;  other  members  will  be  advisory. 

8.  The  primary  responsibility  of  the  Committee  shall  be  to  establish 
guidelines  for  execution  of  quality  medical  care  in  the  office  as 
wel 1  as  the  hospital . 

9.  The  Committee  shall  have  a  permanent  recording  secretary. 

10.  The  Committee  shall  establish  a  standard  coded  system  to  be  used 
by  all  participating  members  for  reporting  procedures  and  diseases 
to  Employers  Insurance  for  reimbursement  and  for  the  purpose  of 
qual ity  review. 

11.  The  PRC  should  operate  separately  from  the  POC,  yet  in  conjunction 
with  the  POC 1 s  activities.    Meetings  shall  be  held  on  a  monthly 
basis,  and  the  minutes  be  presented  to  the  entire  North  Central 
Health  Protection  Plan  Operating  Committee. 

12.  It  shall  be  the  intent  of  the  PRC  to  review  at  periodic  intervals 
every  physician's  hospital  office  practice.    If  any  problems 
develop  as  a  result  of  this  review,  they  shall  be  presented  to 
the  POC. 

13.  The  PRC  is  expected  to  make  an  effort  to  inform  and  educate 
participating  physicians  why  review  is  taking  place. 

14.  Employers  Insurance  of  Wausau  will  continue  to  aid  physicians 

on  the  Committee  with  computer  support,  which  will  enhance  their 
ability  to  conduct  medical  audits  and  peer  review  of  NCHPP.  It 
is  important  that  physicians  understand  the  information  provided 
by  NCHPP. 

15.  The  PRC  shall  maintain  permanent  records,  including  the  monthly 
statistical  reports  furnished  by  Employers  Insurance. 

After  reviewing  PRC  minutes,  the  committee's  activities  emphasize 
communication  with  physicians  who  have  been  reviewed  after  a  problem  has  been 
discovered.    The  letters  usually  ask  the  physician  for  more  information  re- 
garding a  specific  case  or  forward  comments  of  the  review  physician.  The 
committee  has  functioned  at  a  very  high  activity  level,  both  in  reviewing 
profiles  and  communicating  with  NCHPP  physicians. 
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Committee  decision-making  is  described  as  a  consensus  process  in  which 
a  majority  of  committee  members  agree  on  the  gravity  of  a  problem  and  the 
type  of  action  to  be  taken.    The  PRC,  it  was  noted,  benefited  from  the  ser- 
vices of  a  managerial  consultant  who  was  employed  by  the  Wausau  Medical 
Center  to  evaluate  committee  meetings  and  individual  interactions  in  terms 
of  accomplishing  their  objectives.    The  consultant  attended  several  meetings 
and  presented  recommendations  to  the  Center.    This  point  emphasized  both  the 
NCHPP  physicians'  and  HCSD's  attitudes  toward  implementing  an  effective  review 
system. 

The  PRC  also  commits  significant  amounts  of  time  and  resources  (based  on 
information  from  interviews  and  minutes)  to  review  of  inpatient  services. 
The  NCHPP  Hospital  Certification  Program,  as  noted  in  the  documentation, 
includes  hospital  utilization,  extension  requests  and  reviews,  procedures 
for  handling  noncertified  admissions  or  extensions  and  certification  informa- 
tion.   NCHPP  benefits  disallow  diagnostic  admissions,  and  the  nurse  coordinator 
makes  sure  all  questionable  admissions  are  approved  by  a  committee  member.  In 
accordance  with  the  previously  determined  guidelines,  a  physician  of  one  spec- 
ialty reviews  a  fellow  specialist.    PRC  physicians  usually  review  extensions 
of  length  of  stay  and  questions  regarding  admissions.    This  has  proved  to  be 
an  educational  process,  but  now  has  controls  for  noncertified  admissions  or 
extensions.    If  a  physician  makes  a  noncertified  extension  or  admission, 
written  notification  is  made  and  all  charges  paid  from  the  General  Fund. 
The  second  time,  half  of  the  charges  are  paid  from  the  Physicians  Fund  and  half 
from  the  General  Fund;  the  third  time,  all  charges  are  deducted  from  the 
physician's  claim  payment. 

As  well  as  assisting  the  hospital  in  discharge  planning  and  having 
responsibility  for  the  administration  of  certification  procedures,  the 
Nurse  Coordinator  maintains  monthly  statistics  related  to  the  Plan  (with 
regard  to  admissions,  recertif ications  and  extensions).    These  statistics  are 
monitored  by  the  physicians  and  updated  monthly. 

The  POC  reviews  reports  from  the  PRC  and  approves  their  actions,  thus 
the  three  joint  members  provide  some  consistency  in  problem  solving.  The 
POC  represents  the  physicians  of  Marathon  County  and  functions  in  a  policy  role, 
which  includes  several  operational  activities: 


1.  Acts  in  an  administrative  capacity  by  reviewing  operations 
and  by  recommending  or  concluding  changes 

2.  Is  the  policy-making  committee  for  the  administration  of 
NCHPP 

3.  Is  responsible  for  effective  fee  review 


The  Peer  Review  Committee 

1.  Acts  in  a  capacity  of  controlling  quality  and  quantity  of 
medicine  and  will  take  whatever  appropriate  action  is  needed 

2.  Is  responsible  for  review  and  evaluation  of  statistical  data 
pertaining  to  physician  activity  under  the  plan 
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3.  Is  responsible  for  review  and  approval  or  denial  of  hospital 
certification  or  extensions  necessitated  by  the  hospital 
certification  program  (these  cases  are  referred  for  review 
by  the  nurse  coordinator) 

4.  Each  committee  member  is  responsible  for  an  individual  review, 
but  when  he  or  she  is  unable  to  resolve  a  particular  question, 
it  must  be  referred  to  the  entire  committee.    Committee  mem- 
bers' terms  are  staggered  for  one,  two,  and  three  years  to 
maintain  continuity  and  consistency  of  review. 


Data  Processing 

The  NCHPP  information  system  (managed  by  HCSD)  produces  several  reports. 
The  Physician  Summary  Report,  produced  on  a  quarterly  basis,  provides  all 
NCHPP  physicians  a  report  on  their  charges  and  services,  in  addition  to  charge 
and  service  data  for  all  other  physicians  within  the  same  specialty  for  pur- 
poses of  review  and  comparison.    During  interviews,  HCSD  personnel  indicated 
this  may  function  as  a  peer  pressure  mechanism  to  keep  NCHPP  physicians  alert 
to  colleagues'  utilization  patterns.    Three  main  characteristics  of  this  report 
should  be  noted: 

1.  The  reports  are  very  detailed  and  demonstrate  all  procedures 
billed  to  the  Plan  by  a  particular  physician.    Charges  are 
divided  into  four  areas;  surgery,  radiology,  pathology,  and 
medicine. 

2.  Summary  reports  for  various  plans  also  document  outpatient 
charges  in  the  following  areas:    monthly  charges,  number  of 
patients,  services  per  patient,  average  charges,  emergency 
room  charges,  x-ray  tests,  lab  tests  and  physical  therapy 
charges . 

3.  Summary  reports  also  display  out-of-area  referrals  to  non- 
participating  physicians  by  physicians  in  the  Plan.  This 
report  also  tabulates  physicians'  referrals  for  month  and 
year  to  date,  the  person  to  whom  the  referral  was  made  and 
the  cost  of  the  referral  by  service. 

Because  of  data  processing  capabilities,  PRC  and  POC  physicians  have 
access  to  more  than  45  different  programmed  reports.    The  most  important  ones 
are  as  follows: 

1.  Disability  Report  -  all  diagnoses  submitted  to  the  Plan  and  all 
treatments  for  that  diagnosis—by  plan,  by  specialty,  or  by 
physician 

2.  Hospital  Inpatient  and  Outpatient  Report  -  all  charges  rendered 
to  the  Plan  for  inpatients  and  outpatients 

3.  Out-of-Area  Referral  Report 

4.  Charge  and  Service  Recap  Report 

5.  Diagnosis  Profile  Report 

6.  Case  Review 
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Other  reports  are  as  follows: 


1.  Pregummed  labels  for  patient  medical  record 

2.  Cross-reference  cards  for  subscribers  and  dependents 

3.  ^Activity  edit  reports 
Pap  smear  charges 
Charges  exceeding  $500 
Physical  exam  exception  report 

4.  Hospital  payment  voucher 

5.  *Monthly  charge  and  service  analysis  by  physician 

6.  Monthly  payment  voucher 

7.  ^Monthly  report  by  physician  and  by  procedure  code  for  the 

8.  Out-of-area  student  emergencies,  nonstudent  emergencies  and 
existing  disabilities  report 

9.  Hospital  inpatient  physiotherapy  report 

10.  Hospital  confinement  report 

11.  Patient  within  diagnosis  report 

12.  Active  and  terminated  account  listings 

13.  Current  month  addi tions--terminations  report 

14.  Dependent  eligibility  report 

15.  Procedure  code  listings 
Expanded  list 
Condensed  list 

16.  Physician  cross-reference  listing 

17.  Out  of  balance  accounts  report 

18.  Lag  time  report 

19.  Delinquent  posting  report 

20.  Statement  of  charges  report 

21.  Transaction  history  report 

22.  Experience  reports 
By  account 

By  patient 
Termination 

23.  Pure  premium  coverage  report 

24.  Year  end  subscriber  account  summary 

25.  Resurvey  questionnaire 

26.  Daily  trial  balance  report 

27.  Daily  clear  totals  report 

28.  Transaction  adjustment  report 

29.  Practice  master  report 

30.  Numeric  account  report 

31.  Alpha  account  report 

32.  Reserve  report 

33.  Charge  file  total  report 

34.  Expenditure  report 

35.  Physician  payment  voucher 

36.  Out-of-area  emergency  report 

37.  Referral  report 

38.  Treatment  code  report 

*  Used  more  frequently 
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39.  Patient  count  report 

40.  Plan  status  report 

41.  Emergency  room  profile  report 

42.  Plan  analysis  report 

43.  Age  and  sex  table  report 

44.  Dollars  distribution  report 


Peer  Review  Committee  Minutes 

The  on-site  investigators  were  allowed  to  review  PRC  minutes  to  substan- 
tiate committee  activities.    An  abstract  of  the  minutes  is  presented  below 
and  they  substantiate  both  the  peer  review  activities  outlined  in  documents 
and  activities  reported  from  interviews. 

Seven  sets  of  Peer  Review  Committee  meeting  minutes  were  compiled  and 
analyzed  (September  3,  1975  to  March  2,  1976).    Committee  meetings  convened  in 
the  evening  around  7:45  p.m.  and  usually  lasted  about  three  hours.  The 
attendance  rate  was  fair,  running  around  50%  each  meeting  except  for  the 
September  3,  1975,  meeting  when  nine  members  were  present  and  the  February  3, 
1976,  meeting  when  12  were  present.    The  activities  usually  consisted  of  review 
of  the  minutes  of  the  previous  meeting,  discussion  of  account  profiles,  assign- 
ing of  new  reviews,  letters  being  sent  based  on  previous  reviews  and  discussion 
of  letters  received  from  physicians  based  on  previous  reviews.    In  greater 
detail,  the  specific  activities  of  each  of  the  seven  meetings  follow: 

1.  September  3,  1975  7:45  p.m.  to  12:15  a.m.        9  present;  3  absent 

Activities:    Discussion  of  eight  submitted  review  reports 
Minutes  note  ten  reviews  performed 
Eleven  letters  sent  to  providers  based  on  reviews 

2.  October  7,  1975  7:45  p.m.  to  10:25  p.m.        7  present;  6  absent 

Activities:    Minutes  reviewed 

Review  of  letters  from  providers  responding  to  review 
Peer  review  activities  discussed  with  regard  to  account 
profile  responsibility 

3.  November  4,  1975  7:50  p.m.  to  10:20  p.m.        6  present;  6  absent 

Activities:    Report  on  Hospital  Certification  by  one  Peer  Review 

Committee  member  reviewed 
Discussion  of  administrative  input  from  HCSD  into  peer 

review  activities 
Letter  from  one  physician  reviewed  by  Committee  discussed 
Three  account  profiles  discussed 
Ten  new  reviews  assigned 

Four  letters  sent  to  providers  based  on  reviews 
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4.    December  1 ,  1975 


7:50  p.m.  to  10:15  p.m. 


6  present;  6  absent 


Activities:    Motions  to  take  specific  action  on  three  cases 
Six  account  profiles  reviewed 
Discussion  of  grievances  of  two  physicians 
Hospital  Certification  cases  discussed 
Letters  sent  to  seven  physicians  based  on  committee  review 
Discussion  of  letters  from  four  physicians  based  on 
previous  reviews 

5.  January  6,  1976  7:40  p.m.  to  10:50  p.m.  7  present;  5  absent 

Activities:    Hospital  Certification  cases  reviewed 
Mammography  screening  program  reviewed 
Two  out  of  four  letters  received  from  physicians 
responding  to  reviews  discussed 

6.  February  3,  1976  7:40  p.m.  to  11:00  p.m.  12  present;  1  absent 

Activities:    Letters  from  Nurse  Coordinator  discussed 

Evaluations  from  two  review  physicians  discussed 
Six  account  profile  reviews  discussed 
Letters  sent  to  seven  providers  asking  for  information 
on  procedures  and  charges 

7-    March  2,  1976  7:45  p.m.  to  8:55  p.m.  6  present;  6  absent 

Activities:    Discussion  of  review  procedure 

Reviews  assigned  to  Committee  physicians  present 

at  meeting  to  conduct  the  reviews  for  those  not  present 
Discussion  of  letters  sent  to  physicians  based  on  previous 

review 

Hospital  Certification  figures  reviewed 
Written  reports  from  six  physicians  noted 
Ten  account  profiles  discussed 

Eight  letters  sent  to  providers  requiring  explanation  and 
identification  of  charges 


ADDITIONAL  COMMENTS 

No  medical  criteria  were  developed  for  peer  review  activities.  During 
interviews,  NCHPP  personnel  indicated  the  PRC  attempted  to  develop  criteria, 
but  no  consensus  regarding  such  criteria  for  different  specialties  and  indi- 
vidual physicians  could  be  reached.    They  feel  the  present  review  process  is 
funneled  through  enough  checks  and  balances  that  a  review  decision,  once 
reached,  is  a  judicious  medical  opinion  for  a  specific  case  of  care. 

According  to  documentation,  retroactive  denial  was  officially  endorsed  by 
the  participating  NCHPP  physicians  at  the  Marathon  County  Medical  Society  meeting 
of  September,  1975.    The  effective  date  for  this  procedure  was  July  1,  1975, 
with  the  committee  having  the  power  to  deny  charges  back  to  July  1,  1974.  A 
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retroactive  denial  is  a  denial  of  reimbursement  for  services  provided  or  ordered 
by  a  physician  after  money  for  this  service  has  been  provided.    These  denials 
usually  are  the  result  of  a  random  review  of  Account  Profiles  by  the  Peer 
Review  Committee.    The  procedure  for  retroactive  denial  is  as  follows: 

a.  Physician  notified  via  letter  of  denial,  has  30  days  to 
justify  his  or  her  procedure 

b.  If  no  response  after  30  days,  second  letter  sent  requesting 
response 

c.  If  no  response  after  14  days  or  physician  unable  to  justify 
procedure,  denial  remains;  if  committee  approves  justification, 
denial  cancelled.    All  appeals  are  sent  to  the  POC.  Study 
staff  were  unable  to  determine  the  effect  of  this  program  at 
the  time  of  the  site  visit. 

NCHPP's  program  had  several  unique  characteristics: 

1.  Physicians  serving  on  the  committees  do  so  without  charge, 
except  for  a  small  reimbursement  provided  to  a  physician  who 
reviews  a  specific  case  for  certification  or  presents  it  to 
the  Committee. 

2.  Physicians'  behavior  may  be  affected  by  the  knowledge  that 
their  procedures  and  diagnoses  can  be  closely  monitored  at 
any  given  time  by  the  Committee. 

3.  Each  contract  is  with  an  individual  physician  rather  than  a 
group.    Each  physician  makes  his  or  her  own  decision  to  parti- 
cipate and  accept  the  responsibilities  according  to  the  agree- 
ment. 

4.  The  fee-for-service  operation  rewards  a  physician  for  individual 
ability  and  initiative. 

5.  The  Plan  is  operated  solely  on  private  resources. 

6.  The  Plan  controls  its  patient  and  physician  population.  There 
is  no  open  enrollment. 

7.  Reviewed  profiles  are  of  entire  families  for  whom  services 
have  been  provided  for  a  period  of  12  to  24  months. 


55 


230-045  O  -  77  -  5 


SECTION  III 


DELIVERY       SYSTEM  CHARACTERISTICS 


The  Health  Care  System  Division  within  Employers  Mutual  Liability  Insurance 
Company  of  Wausau,  Wisconsin,  is  a  functioning  HMO  with  contractual  relation- 
ships with  five  Individual  Practice  Associations  (IPA's).    According  to  docu- 
mentation, this  plan  does  the  following: 

1.  Preserves  the  fee-for-service  approach. 

2.  Accommodates  all  types  of  medical  practices,  from  solo  practices 
to  clinic  practice. 

3.  Affords  those  enrolled  in  the  Plan  free  choice  of  participating 
physicians. 

4.  Pays  in  full  nearly  all  medical  expenses  for  both  inpatient  and 
outpatient  care. 

5.  Pays  for  preventive  as  well  as  curative  services. 

According  to  the  interview,  HCSD  has  developmental,  conceptual,  and 
operational  responsibilities  to  the  Health  Protection  Plans.  Initial  involvement 
in  a  community  begins  with  the  development  of  both  medical  provider  support 
and  industry  support.    Representatives  from  HCSD  speak  to  key  physicians,  the 
local  medical  society,  and  community  leaders  about  the  important  conceptual 
aspects  such  as  risk  sharing,  concurrent  hospital  review,  ambulatory  peer 
review,  guaranteed  fees  and  charges,  and  computerized  administration. 

Each  HPP  (IPA)  is  separately  developed.    Marketing  activities  are  based 
on  the  number  of  physicians  and  employers  within  a  specific  geographic  region- 
such  as  a  county  medical  society  area.    HCSD  tries  to  maintain  a  high  level 
of  interaction  between  physicians,  institutional  providers  and  industry. 


SERVICES 

The  Plan  pays  in  full  practically  all  medical  expenses  of  both  inpatient 
and  outpatient  care  given  at  a  particular  physician's  office  or  clinic  and  at 
a  particular  hospital.    Acute,  preventive,  diagnostic  and  ambulatory  care 
services  are  covered  by  the  Plan.    The  maximum  benefit  per  person  is  250,000 
dol lars . 

With  regard  to  preventive  care,  benefits  include  payment  in  full  for 
office  visits,  immunizations,  diagnostic  testing,  medical  eye  exams,  routine 
physical  exams  (within  a  specified  time  period)  and  pap  smears,  which  are 
available  annually.    Twenty-four  hour  emergency  room  care  is  also  covered. 
Mental  health  services  occurring  in  the  hospital  are  paid  in  full  while  those 
occurring  outside  the  hospital  are  covered  to  $1,000  per  year. 

It  is  noted  in  the  documentation  that  the  Plan  was  initially  available  to 
eligible  full-time  employees  of  existing  groups  of  25  or  more  employees  and 
their  eligible  dependents.    These  dependents  can  be  a  spouse  and  unmarried 
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dependent  children  up  to  19  years  of  age  (if  students,  eligible  until  25  years 
of  age).    Each  enrolled  employee  has  the  free  choice  of  selecting  a  partici- 
pating physician. 


STAFF 

Approximately  96  physicians  are  participating  in  the  Plan.  The  role  of 
each  physician  is: 

1.  To  assure  adequate  manpower 

2.  To  provide  utilization  review  and  peer  review 

3.  To  provide  review  of  fee  schedules  submitted  by  particular 
physicians  in  the  development  of  relative  uniformity  by  specialty 

4.  To  abide  by  decision  of  the  physician  committee,  which  represents 
all  physicians  in  making  necessary  decisions  and  providing 
guidance  for  the  success  of  the  Plan 

The  entire  HCSD  is  managed  by  a  Health  Care  Committee  within  Employers 
of  Wausau.    Five  senior  vice  presidents  evaluate  the  total  program.  They 
report  directly  to  the  president  of  the  company.    An  assistant  vice  president 
is  in  charge  of  HCSD.    In  addition  to  the  assistant  vice  president,  the  staff 
include: 

Manager,  health  care  systems  operations 
Three  HPP  administrators 
One  research  underwriter 

One  supervisor  responsible  for  the  activities  of  the  data  clerks 

and  claim  clerks 
One  financial  reporter 
One  Milwaukee  administrative  assistant 
Four  payment  processors 
Five  telecommunicators 
One  clerk  typist 

Two  administrative  reports  processors 


57 


SECTION  IV 


H  C  MS      STUDY  INFORMATION 


NCHPP  (HCSD)  was  considered  for  inclusion  in  the  study  on  recommendation 
of  a  Project  Advisory  Board  member.    The  assistant  vice  president  of  HCSD  was 
contacted  by  study  staff  on  February  10,  1976.    He  was  introduced  to  the  pro- 
ject and  expressed  interest  in  participating.    Correspondence  was  exchanged 
and  HCSD  agreed  to  participate  in  the  study  on  March  9,  1976,  during  a  telephone 
discussion  with  the  administrator  of  NCHPP. 

HCSD  forwarded  initial  information  (March  29,  1976)  to  study  staff,  which 
included  the  following  documents: 

1.  Three-page  internal  correspondence  (March  29,  1976) 

2.  Health  Protection  Plans  Organization  (3-page  narrative  statement) 

3.  Health  Protection  Plans  Organization  Chart  (copyright,  1976) 

4.  Definition:    Peer  Review  Committee  of  the  NCHPP 

5.  Relationship  and  Responsibilities  of  the  Operating  Committee  and 
the  Peer  Review  Committee 

6.  Guidelines  for  the  Peer  Review  Committee  of  the  NCHPP 

7.  NCHPP  -  Peer  Review  Summary 

8.  NCHPP:    Major  features  of  the  Plan  (January  1,  1974) 

9.  NCHPP:    Hospital  Certification  Program  (January  1,  1974) 

10.  Peer  Review  Report  (March  1,  1976) 

11.  NCHPP  Patient  Profile  (February  13,  1976) 

12.  NCHPP  Statistical  Summary  Sheet  (January  to  April,  1976,  39  pages) 

13.  NCHPP  encounter  forms 

14.  NCHPP  coding  sheet  (computer  input  document) 

15.  NCHPP  Physicians  Participation  Agreement  (November,  1972) 

16.  List  of  NCHPP  Computer  Reports 

This  documentation  outlined  the  organization,  program  elements  and 
objectives  of  NCHPP  in  relation  to  HCSD  within  Employers  of  Wausau. 

The  dates  of  the  site  visit  were  April  27  and  28,  1976.    A  significant 
amount  of  information  was  collected  during  the  site  visit,  but  the  interviews 
were  not  taped.    Questions  were  addressed  to  the  following  personnel: 

1.  Assistant  Vice  President  of  HCSD 

2.  Manager  of  HCSD  Operations 

3.  Chairman  of  Physicians'  Peer  Review  and  member  of  Operating  Committee 

4.  Member  of  Peer  Review  Committee 

5.  Administrator  of  NCHPP 

6.  Administrator  of  Eau  Claire  HPP 

7.  Research  Underwriter 

8.  Administrator  of  Milwaukee  HPP 

9.  Assistant  -  General  Counsel 
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The  topics  covered  during  the  interview  were: 


1.  Administrative  structure  of  HCSD 

2.  Historical  development  of  HCSD 

3.  Initial  development  of  IPAs  by  Employers  Insurance  of  Wausau 

4.  Philosophy  and  approach  of  HCSD  with  regard  to  the  delivery  of 
medical  services 

5.  Marketing  activities  of  HCSD  and  IPAs 

6.  Unique  characteristics  of  NCHPP 

7.  Goals  of  the  review  process  of  NCHPP 

8.  Peer  Review  Committee  activities  and  functions 

9.  Interrelationship  between  Peer  Review  Committee  and  Physician 
Operating  Committee 

10.  Decision-making  by  Peer  Review  Committee 

11.  Encounter  form  used  at  NCHPP 

12.  Accuracy  and  validity  with  regard  to  coding  and  data  accuracy 

13.  Fee  schedules 

14.  Account  profile  review 

15.  Physician  profile  review 

In  addition  to  the  interview,  the  Peer  Review  Physicians  performed  sample 
account  profile  reviews  for  the  study  staff. 
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MATTHEW  THORNTON  HEALTH  PLAN 


Nashua,  New  Hampshire 


SECTION  I 
INTRODUCTION 


Matthew  Thornton  Health  Plan  (MTHP)  is  a  prepaid  nonprofit  group  practice 
operating  with  Blue  Cross  support  in  Nashua,  New  Hampshire.    The  clinic  has 
been  in  operation  since  November,  1971,  and  the  quality  assurance  program  (QAP) 
has  been  operational  for  approximately  18  months. 

The  clinic  provides  eight  major  services  to  approximately  29,000  persons 
and  is  staffed  by  32.5  FTE  (full  time  equivalent)  medical  employees  and  33.5 
FTE  administrative  employees. 

MTHP  has  two  distinct  organizational  structures:    (a)  administrative  staff, 
and  (b)  medical  staff,  composed  of  physicians,  physician  assistants  and  nurse 
practitioners.    A  lay  board  of  directors  has  overall  responsibility  for  the 
operation  of  plans,  with  the  medical  director  reporting  directly  to  the  board. 
Under  the  medical  director  is  the  executive  director,  who  administers  finance, 
marketing,  ancillary  services,  personnel  and  plan  management. 

MTHP  does  not  have  a  highly  formal  quality  assurance  program,  but  does 
conduct  clerical  and  peer  review  of  medical  records,  as  well  as  holding  in- 
formal peer  review  meetings.    MTHP  was  included  in  the  study  because  of  active 
development  of  quality  assurance  elements  and  because  of  its  success  in  using 
the  problem  oriented  medical  record  (POMR). 
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SECTION  II 

QUALITY       ASSURANCE  PROGRAM 


Before  its  formal  inclusion  in  the  study,  a  Matthew  Thornton  Health 
Plan  physician  suggested  that  MTHP  should  be  included  because  it  was  in  the 
process  of  developing  a  formal  quality  assurance  program  based  on  a  fairly 
large  experience  of  informal  activities. 

MTHP  personnel  specified  several  factors  responsible  for  the  development 
of  a  quality  assurance  program  (QAP): 

1.  The  key  factor  in  developing  any  assessment  program  is  the 
documentation  of  specific  physician-patient  encounters;  the 
main  emphasis  over  the  last  two  years  has  been  to  assure  the 
proper  use  of  the  problem  oriented  medical  record  (POMR). 

2.  Since  MTHP  is  the  first  prepaid  health  plan  in  the  area  and 
the  largest  group  practice,  the  group  felt  it  was  important 
to  formalize  audit  procedures  as  a  marketing  point  with  the 
community. 

3.  Both  the  medical  director  and  the  quality  assurance  director 
were  committed  to  formalizing  procedures  and  convincing  the 
group  of  physicians  and  the  Board  of  Trustees  of  their 
importance. 

4.  The  importance  of  continuing  education  has  made  quality  assur- 
ance an  essential  ingredient  in  developing  education  programs 
for  the  whole  professional  staff. 

5.  MTHP  staff  felt  they  should  have  some  assessment  of  the  quality 
of  their  medical  care  to  make  needed  adjustments  in  techniques 
and  procedures  that  improve  care. 

MTHP  quality  assurance  program  does  not  include  a  systematic  monitoring 
of  components  in  the  medical  system.  MTHP  personnel  said  a  "constant  monitor- 
ing program"  for  particular  disease  conditions  was  not  feasible  or  necessary 
at  this  particular  time.  Formal  quality  assurance  activities  have  consisted 
of  surgical  and  hospitalization  reviews.  MTHP  personnel  said  that  since  MTHP 
has  been  in  operation  for  five  years,  resources  necessary  for  formal  reviews 
have  focused  on  the  inpatient  sector. 

No  documents  are  available  describing  the  development  and  implementation 
of  the  QAP.    HCMS  did  receive  a  lengthy  correspondence  from  an  MTHP  internist 
outlining  key  features  of  the  peer  review  program,  and  indicating  the  develop- 
mental nature  of  the  program.    Program  information  was  also  collected  during  the 
on-site  investigation. 

The  administrative  staff  of  MTHP  has  little  input  into  direction  of  peer 
review  activities.    By  reporting  to  the  board  of  directors  the  medical  director 
has  developed  a  good  source  of  support  and  has  convinced  the  Board  it  is 
worthwhile  and  cost  effective  to  develop  formal  quality  assurance  programs. 
The  Board  of  Trustees  passed  a  formal  resolution  in  support  of  the  embryonic 


61 


peer  review  activities.    Specific  decision  making  for  quality  assurance 
emanates  from  the  medical  director  and  depends  on  his  relationship  with  the 
entire  medical  staff.    Goals  and  objectives  were  not  set  down  formally,  but 
two  general  principles  were  agreed  on  during  group  discussions  between 
physicians  and  physician  assistants,  who  play  a  major  role  in  delivering 
services  at  MTHP: 

1.  The  program  should  allow  for  adjustments  in  techniques  and 
procedures  which  will  improve  care. 

2.  Quality  of  care  assessment  should  be  an  essential  and  integral 
part  of  MTHP  efforts  for  continuing  education  of  the  entire 
professional  staff. 

Specific  systematic  quality  assurance  methods  have  not  yet  been  selected 
by  MTHP;  however,  various  techniques  and  procedures  have  been  adapted  to  fit 
MTHP's  needs. 


OPERATIONAL  ACTIVITIES 

Quality  review  at  MTHP  has  two  major  aspects,  the  clerical  POMR  audit,  and 
the  peer  review  audit. 


Clerical  Audit  of  POMR 

The  importance  of  the  MTHP  audit  system  is  the  investment  of  resources 
in  systematically  recording  and  typing  encounter  information  in  the  medical 
record.    The  emphasis  at  MTHP  has  been  to  improve  the  documentation  of  care, 
"since  any  quality  assurance  program  will  only  be  as  good  as  the  information 
in  the  record."    All  data  placed  in  medical  records  are  typed. 

The  clerical  audit  of  the  POMR  is  the  most  consistent  and  best  developed 
procedure  at  MTHP.    A  data  base  has  been  established  and  each  physician  or 
physician's  assistant  is  required  to  complete  it  whenever  a  patient  has  a 
"complete  physical."    This  system  is  monitored  with  monthly  assessments  of 
whether  all  parameters  were  met. 

Because  the  use  of  the  POMR  requires  highly  specific  information,  the 
audit  procedure  identifies  whether  specific  data  elements  are  present  in  the 
record. 

POMR  audits  are  the  main  focus  of  activity  at  present.    They  are  con- 
ducted by  an  administrative  staff  person,  whose  responsibility  includes  trans- 
cription of  all  medical  records.    The  results  are  computed  for  each  provider. 
A  tally  sheet  compiles  the  provider's  name,  the  number  of  chart  audits,  the 
number  of  omissions  and  the  accuracy  (percentage  of  charts  with  no  omissions). 
Results  are  sent  to  the  medical  director,  who  uses  audit  results  for  decision- 
making.   The  assessment  procedures  are  as  follows: 
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1.  Patients  are  classified  in  one  of  four  ways--males  over  40, 
males  under  40,  females  over  40,  and  females  under  40. 

2.  For  each  classification,  audit  values  (also  called  data  elements) 
were  developed  (Figure  1). 

3.  Every  month  an  audit  is  conducted  on  data  bases  for  all  physicians. 

4.  The  charts  that  are  pulled  are  the  POMRs  of  the  complete  data 
bases  that  were  conducted  the  previous  month  (i.e.,  in  January  all 
data  bases  completed  in  December  are  audited). 

5.  The  audit  values  listed  above  must  be  included  in  the  data  base  for 
patients  in  the  four  categories. 

6.  The  monthly  sample  of  charts  is  then  audited  by  the  administrative 
assistant. 

7.  Results  of  the  audit  are  reported  in  terms  of  the  frequency  of 
omission  of  the  audit  values. 

8.  Results  are  given  to  the  medical  director. 

9.  It  is  up  to  the  medical  director  to  take  action  with  the  individual 
physicians  if  they  were  omitting  specific  audit  values,  but  the 
medical  director  will  have  a  patient  return  to  the  clinic  for 
completion  of  the  initial  assessment. 

The  administrative  assistant  responsible  for  auditing  said  approximately 
107  charts  are  audited  monthly.    Omissions  are  sent  to  the  medical  director 
who  has  full  discretion  and  responsibility  for  taking  action.    Most  audits 
are  done  on  initial  health  evaluations,  which  in  a  P0MR  system  are  the  foundation 
for  the  data  base,  and  for  other  care  episodes  and  encounters. 

MTHP  personnel  indicated  clerical  audits  are  considered  the  first  stage 
in  developing  disease-  or  condition-specific  audits. 


Peer  Review  Functions 

Peer  review  is  the  responsibility  of  the  entire  clinical  staff.  When 
peer  review  meetings  are  held  to  discuss  quality  assurance  issues,  all  MTHP 
providers,  physician  assistants,  nurses,  and  key  administrative  personnel 
attend.    Site  investigators  were  allowed  to  attend  one  such  meeting,  and  made 
these  observations: 

a.  Discussions  follow  a  general  outline,  but  are  fairly 
unstructured 

b.  Rate  of  attendance  of  physicians  and  physician  assistants  is 
high 

c.  The  primary  physician  presents  topic  cases  (in  this  instance 
three  recent  deaths),  bringing  documentation  into  the 
meeting 

d.  Most  physicians  attending  made  comments  on  cases 

e.  No  decisions  were  discussed  in  the  meeting 

f.  The  meetings  seemed  to  have  educational  value 

g.  Minutes  were  taken  at  the  meeting 
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MTHP  personnel  stated  the  three  areas  of  peer  review  were  not  implemented 
in  a  systematic  manner,  and  would  remain  informal  until  clinical  staff  could 
devote  more  time  to  review,  or  until  discovery  of  critical  issues  that  require 
systematic  monitoring.    The  three  kinds  of  peer  review  are  death,  problem  case, 
and  general,  defined  as  follows: 

1.  Death  Review  is  conducted  approximately  every  three  months,  but 
seems  to  be  irregular. 

All  deaths  in  the  practice  are  reviewed  in  terms  of  outcome  and 
the  procedure  of  the  medical  care.  This  is  usually  one-to-one, 
another  physician  doing  the  reviewing. 

The  peer  review  meeting  attended  by  study  staff  dealt  with  two 
death  reviews.    The  case  discussions  seemed  to  accent  general 
education  for  the  MTHP  providers. 

2.  Problem  Case  Review  is  designed  to  reveal  problems  in  two  areas: 

a.  Patient  complications  in  management  of  a  particular 
problem  or  condition 

b.  Exceptions  noted  in  the  medical  record  and  medical 
problems  or  both 

Each  physician  is  asked  (via  memoranda)  to  "select  one  patient" 
whose  medical  record  documents  a  difficult  or  interesting 
treatment  diagnosis.    A  peer  review  audit  sheet  accompanies 
this  memo,  listing  15  questions  to  be  answered  by  the  physician 
about  the  problem  case. 

MTHP  attempts  to  conduct  this  review  every  three  months  for  16 
to  20  cases.    No  minimum  or  maximum  number  of  cases  is  required. 

After  the  coordinating  physician  receives  the  medical  staff 
responses,  he  will  route  the  medical  record  to  an  independent 
physician  for  implicit  review. 

Peer  review  meetings  are  then  held  to  discuss  the  cases, 
especially  medical  management  problems.    The  coordinating 
physician  reviews  the  cases  before  review  meetings  to 
facilitate  the  procedure.    Two  results  of  meetings  are 
possible: 

1)  Education  activities  could  be  implemented 

2)  Reports  on  the  total  management  of  patients  could 

be  discussed  and  possibly  improved  (because  patients 
see  a  variety  of  MTHP  physicians) 

3.  General  Reviews  are  topic-oriented  audits,  wherein  a  specific 
disease  entity  is  chosen,  data  are  collected  and  analyzed  and 
results  are  discussed  in  a  peer  review  meeting.    HCMS  staff 
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received  evidence  of  a  hypertension  audit  conducted  in 
September,  1975.    The  focus  of  this  audit  was  to  look  at 
a  large  group  of  hypertensive  patients  to  see  if  a  few 
parameters  were  being  met  to  assess  the  quality  of  long- 
term  management  of  a  chronic  condition. 

The  audit  was  conducted  in  the  following  manner: 

a.  More  than  100  charts  with  the  terminal  number  98  were 
chosen;  out  of  this  group,  14  charts  contained  a 
diagnosis  of  hypertension. 

b.  Charts  from  the  last  computer  printout  for  hyper- 
tension were  checked.    Again  the  selection  of  charts 
was  by  terminal  digits  derived  from  a  list  of  random 
numbers. 

c.  The  charts  were  pulled  and  values  to  be  checked  were: 

1)  Was  patient  seen  in  the  last  six  months  for 
hypertension? 

2)  Was  the  average  of  the  last  3  systolic  pressures  less 
than  170?    (The  data  in  this  category  reflects,  in 
most  cases,  the  last  and  only  BP  reading  of  the 
record. ) 

3)  Was  the  average  of  the  last  3  diastolic  pressures  less 
than  100?    (In  most  cases  only  one  value  was  avail- 
able, and  that  value  was  used.) 

The  10  items  listed  below  were  the  audit  elements  on  the  79  hyper- 
tension charts  selected: 

MTHP  chart  number 

Date  of  last  blood  pressure  recording  at  MTHP 
Last  blood  pressure  recorded  at  MTHP 
Date  last  seen  at  MTHP 

Primary  provider  for  blood  pressure  follow-up 

Last  visit  to  MTHP  specifically  for  blood  pressure 

Provider  and  reason  last  seen  at  MTHP 

Last  instructions  received  at  visit  to  MTHP 

Blood  pressure  medication(s)  prescribed 

Blood  pressure  flow  sheet  completed 

d.  The  administrative  assistant  (who  also  conducts  chart  audits) 
conducted  this  peer  review  audit  on  hypertension. 

e.  The  79  charts  were  given  a  distribution  in  terms  of  the  values 
that  were  and  the  values  that  were  not  met. 

f.  Fifty-four  charts  examined  satisfactorily  met  the  criteria  being 
checked  for.    Twenty-five  charts  were  presented  for  peer  review 
because  of  deficient  criteria  findings. 
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Investigators  found  no  documentation  regarding  specific 
activities  in  these  three  possible  outcome  areas.  Study 
staff  received  no  report  regarding  the  use  of  results. 
The  peer  review  meeting,  where  the  hypertensive  charts 
were  discussed,  was  not  documented  in  minutes. 


ADDITIONAL  COMMENTS 

The  medical  director  noted  that  a  small  group  practice  encourages  constant 
interaction  among  physicians  and  therefore  requires  a  less  formal  assessment 
program  than  a  large  group.    Formal  and  informal  perceptions  of  the  quality 
of  service  delivered  are  very  close  at  MTHP,  and  formal  quality  assessment 
procedures  are  important,  but  not  as  important  as: 

1.  The  type  of  physician  hired  to  come  into  the  group  practice 

2.  Physicians  sharing  charts  and  thereby  monitoring  what  type 
of  care  is  delivered  by  one  another 

There  seems  to  be  excellent  commitment  on  the  part  of  MTHP  staff  to  formal 
quality  assessment.    However,  due  to  the  rapid  growth  of  the  practice,  peer 
review  is  conducted  during  lunch  meetings  and  on  voluntary  time.    The  amount  of 
time  the  physicians  now  have  to  conduct  reviews  and  take  part  in  the  activities 
is  limited. 

Also  due  to  rapid  growth,  MTHP  has  been  faced  with  pressing  administrative 
and  management  problems.    It  has  only  been  in  the  last  six  to  twelve  months 
that  effective  work  on  starting  a  QAP  has  begun. 

A  unique  aspect  of  the  MTHP  quality  review  is  the  dedicated  and  success- 
ful use  of  the  POMR. 

MTHP  is  in  the  process  of  establishing  a  more  comprehensive  education 
program  for  diabetic  patients.    One  nurse  has  become  very  active  in  it  and  is 
setting  up  a  pre-and-post  test  to  be  given  to  patients  to  monitor  the  effect- 
iveness of  the  diabetic  education  program. 
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SECTION  III 

DELIVERY       SYSTEM  CHARACTERISTICS 


MTHP  is  a  prepaid  group  practice  offering  eight  general  categories  of 
services:    internal  medicine,  pediatrics,  general  and  pediatric  surgery, 
obstetrics  and  gynecology,  psychiatry,  dietetics,  laboratory  and  x-ray. 
Utilization  rates  by  department  are  as  follows: 


Internal  Medicine 

51% 

Pediatrics 

29% 

General  &  Pediatric  Surgery 

9% 

Obstetrics  &  Gynecology 

7% 

Psychiatry 

3% 

Dietetics 

1% 

There  are  12.5  FTE  physicians,  eight  physician  assistants  and  12  nursing 
aides  and  technicians.    The  distribution  of  salaried  physician  employees  at 
MTHP  Inc.  is  as  follows: 


Internal  Medicine  4 

General  Surgeons  2 

OB-GYN  2 

Pediatricians  3 

Part-time  Psychiatrist  1 

Medical  Resident  1 


Pediatric  Nurse  Practitioner  1 

Medex-Physician  Assistants  5 

OB-GYN  Nurse  Practitioner  1 

Part-time  Social  Worker  1 

Part-time  Dietitian  1 


Other  services  include  home  care,  acute  ambulatory  care  at  the  medical 
center,  hospital  services  (both  physician  services  and  hospital  benefits) 
outside  referrals,  maternity  care,  ambulatory  services,  mental  health  benefits 
and  major  medical  coverage. 

Forty  percent  of  the  services  are  provided  in  the  morning  and  sixty 
percent  are  provided  in  the  afternoon  and  evening. 

MTHP  is  affiliated  with  medical  organizations  and  schools  in  these  ways: 

1.  Physicians  have  teaching  appointments  at  Dartmouth  Medical 
School,  Tufts  Medical  School  and  several  teaching  hospitals 
in  Boston. 

2.  Two  of  the  MTHP  physicians  are  members  of  the  New  Hampshire 
Medical  Society. 

3.  One  physician  is  on  a  committee  for  the  New  Hampshire  Medical 
Care  Foundation  (designated  PSRO  for  New  Hampshire). 

4.  All  of  the  physicians  and  the  physicians'  assistants  are  licensed 
with  the  New  Hampshire  Board  of  Registration  in  Medicine. 

There  are  1,200  members  of  the  prepaid  plan  and  28,000  active  fee-for- 
service  patients  registered.  (An  active  patient  is  one  who  has  been  to  the 
clinic  in  the  last  18  months).    In  the  1975  calendar  year,  65,000  to  75,000 
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visits  were  recorded  at  MTHP.    The  majority  of  the  patients  live  within  a 
15-mile  radius  of  the  clinic,  but  some  live  as  far  as  30  miles  away. 

MTHP  has  a  service  area  population  of  136,000.  The  average  family  size 
is  4.03  and  the  median  income  is  $10,960. 

Payment  for  services  at  MTHP  is  from  the  following  sources: 

70.6%  Third  Party  Payers 

16.2%  Fee-for-service  (out  of  pocket) 

9.0%  Medicaid  and  Medicare 

4.3%  Strictly  prepaid 

The  third  party  category  is  funded  primarily  through  employer-employee 
groups  by  Blue  Cross  and  is  funded  on  a  prepayment  capitation  basis. 

MTHP  has  received  federal  funding  for  specific  HMO  projects  and  early 
development. 

There  are  two  distinct  organizational  structures  at  MTHP,  administrative 
staff  and  medical  staff.    The  Board  of  Directors  has  overall  responsibility 
for  operation  of  the  plan.    It  is  composed  entirely  of  citizens  of  the  Nashua 
area  community  (no  physicians).    The  medical  director  reports  directly  to  the 
Board  of  Directors.    Under  the  medical  director  are  the  executive  director, 
who  is  head  of  the  administrative  staff,  and  the  chief  of  professional  ser- 
vices, laboratory,  X-ray,  etc.    Directly  below  the  executive  director  are 
the  operations  manager,  the  finance  manager  and  the  prepayment  department 
manager.    Under  the  operations  manager  are  the  receptionists  for  telephone 
and  appointments,  and  the  medical  record  and  typing  supervisor.    Under  the 
financial  manager  are  the  accounting,  billing,  credit  and  collections  opera- 
tions. 

MTHP  uses  a  problem  oriented  medical  record  (P0MR).  The  medical  records 
include: 

Data  Elements:    Identification  sheet  and  insurance  coverage 
Patient  ID  (coding  system) 
Provider  ID  (coding  system) 

Clinical  Data:    Diagnosis,  Treatment,  Follow-up 
Lab  sheet/X-ray 
Ancillary  services 
Drugs 

Consultation  and  Referrals 

Besides  recording  information  in  the  medical  record,  individual  data 
sheets  are  kept  in  one  book  for  blood  pressure  readings  on  hypertensive  pat- 
ients and  a  central  tumor  registry  for  all  patients  with  cancer  or  malignancy 
is  kept  for  follow-up  procedures. 
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All  entries  by  providers  are  dictated  and  typed  into  the  chart.  X-ray 
slips  are  placed  in  the  chart  and  entered  on  a  lab  flowsheet  by  the  provider. 
Lab  data  are  entered  on  a  lab  flowsheet  by  office  personnel. 

At  present,  MTHP  uses  an  outside  computer  billing  service.  This  service 
can  document  utilization  by  department,  by  diagnosis  and  by  procedure. 
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SECTION  IV 

HC  MS     STUDY  INFORMATION 


Prior  to  the  site  visit,  HCMS  staff  received  the  following  documents: 

1.  Data  Collection  Instrument  1  and  Section  2,  Part  1  completed 

2.  Summary  of  Grant  Support  and  Capital  Financing 

3.  Memoranda  that  included  an  outline  of  a  peer  review 
meeting  and  results  of  the  hypertension  study 

4.  Reports  and  memos  identifying  results  of  the  data  base 
chart  review 

5.  Minutes  of  a  peer  review  meeting  (November  14,  1975) 

6.  Peer  review  outline  sheet 

7.  Correspondence 

8.  Income  Statement  for  MTHP,  Inc.,  November  1975 

9.  Section  from  grant  application  with  demographic  data  regarding 
the  patient  population  and  service  area;  "Denson  Tables"  pro- 
viding a  breakdown  of  fee-for-service  and  prepaid  patient 
population 

10.  Audit  values  for  the  four  classifications  medical  records  are 
are  divided  into 

11.  MTHP  organizational  charts  for  administrative  staff  and 
medical  staff 

12.  Billing  Forms  for  Internal  Medicine  and  Psychiatry,  Surgical, 
Obstetrics  and  Gynecology,  Pediatrics  and  Adolescent,  Medicine 
and  Allergy 

13.  Pamphlet  on  MTHP,  Inc.,  "A  Prepaid  Group  Practice" 

14.  Patient  Registration  Form 

15.  Receipt  of  payment  (Payment  Slip) 

Because  no  documents  explaining  the  development  and  implementation  of  the 
chart  audit  procedure  or  the  peer  audit  procedure  were  received,  the  site  visit 
was  used  to  obtain  both  documented  program  outlines  and  operational  components 
of  the  program. 

The  four  persons  interviewed  were  the  medical  director,  an  internist,  the 
administrative  assistant  and  the  operations  manager. 

The  medical  director  has  overall  responsibility  for  the  number  of  audits 
conducted,  and  for  the  assessment  and  assurance  of  quality  medical  care. 

A  specialist  in  internal  medicine  is  responsible  for  the  routine  organiza- 
tion and  implementation  of  quality  assurance  activities. 

The  administrative  assistant  is  responsible  for  handling  the  data  proces- 
sing for  all  the  problem  oriented  medical  records  (POMR)  and  the  data  collec- 
tion and  administration  of  the  audits.    She  reports  directly  to  the  internist 
and  to  the  medical  director. 
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The  operations  manager  is  responsible  for  the  organization  components 
of  the  audits,  organizes  peer  review  meetings,  and  makes  certain  documenta- 
tion go  in  and  out  of  those  meetings  and  is  reported  properly  to  both 
medical  and  administrative  personnel. 
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ST.  LOUIS  PARK  MEDICAL  CENTER 
Minneapolis,  Minnesota 


SECTION  I 
INTRODUCTION 


St.  Louis  Park  Medical  Center  (SLPMC)  is  a  large  multi-specialty  group 
practice,  operating  as  a  profit-making  business  trust.    SLPMC  is  owned  and 
governed  by  physician  beneficiaries  who  employ  the  entire  administrative  and 
auxiliary  staff,  as  well  as  nonbeneficiary  physicians. 

SLPMC  has  been  in  operation  since  1951,  providing  primary,  secondary  and 
tertiary  patient  care  through  30  specialty  services  in  17  departments.  The 
present  quality  assurance  program  (QAP)  was  initiated  in  October,  1975. 

SLPMC 's  general  organizational  structure  consists  of  a  board  of  trustees, 
a  president,  a  medical  director,  and  a  business  administrator.    The  medical 
director  reports  directly  to  the  president,  and  clinical  department  chiefs 
report  directly  to  the  medical  director. 

Quality  assurance  and  review  activities  at  St.  Louis  Park  are  being  im- 
plemented in  a  four-year  program  sponsored  by  funds  from  SLPMC,  INTERSTUDY 
and  W.K.  Kellogg  Foundation.    A  $306,000  grant  from  the  W.K.  Kellogg  Founda- 
tion supports  approximately  one-half  of  the  four-year  program  cost,  and 
specifically  supports  efforts  to  extensively  document  and  disseminate  the  imple- 
mentation of  the  program. 

The  QAP  staff  consists  of  a  project  director,  who  practices  one-half  time 
in  pediatrics,  a  full  time  executive  director,  a  part-time  biostatistician, 
one  full  time  medical  records  technician  and  two  volunteer  doctoral  students 
studying  diagnostic  and  problem  codes.    The  QAP  has  two  advisory  panels: 

1.  An  advisory  board  comprised  of  ten  SLPMC  physicians  and  the 
assistant  administrator 

2.  An  advisory  panel  comprised  of  individuals  involved  with  quality 
assessment  activities  throughout  the  country. 

The  SLPMC  program  addresses  operational  research-related  activities  by 
establishing  a  systematic  administrative  program  for  implementing  a  problem- 
identification  type  of  quality  assurance  activity.    Because  the  primary  goal 
is  to  put  into  operation  a  quality  assurance  methodology  in  administrative  as 
well  as  medical  departments,  SLPMC  was  included  in  the  study. 

Information  presented  here  reflects  approximately  four  months  of  operational 
quality  assurance  activities,  as  of  the  site  visit  date. 
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SECTION  II 


QUALITY     ASSURANCE  PROGRAM 


This  entire  section  will  describe  SLPMC's  planned  and  implemented  quality 
assurance  activities  in  three  categories: 

1.  Background  and  planned  activities 

2.  Specific  developmental  procedures  conducted  by  quality  assurance 
staff  personnel 

3.  Operational  activities  for  Phases  I  and  II 


BACKGROUND  AND  PLANNED  ACTIVITIES 

Formal  quality  assessment  activities  began  at  SLPMC  in  1971  when  the 
medical  director  initiated  a  peer  review-medical  audit  system.    Because  it 
was  too  costly  and  not  beneficial  enough  to  the  practice,  the  system  was  dis- 
continued.   After  a  period  of  informal  activity,  the  present  quality  assurance 
program  (QAP)  was  initiated  because  of  four  identifiable  factors: 

1.  The  initiative,  experience  and  availability  of  the  project  director 
who  came  to  SLPMC  in  August,  1975,  from  the  Bureau  of  Community 
Health  Services,  DHEW 

2.  The  willingness  of  the  trustees  and  physicians  at  SLPMC  to  pioneer 
in  a  sensitive  area,  and  to  make  a  commitment  to  fund  the 
activities 

3.  The  willingness  of  INTERSTUDY  to  accept  responsibility  for  the  effort 
and  to  fund  part  of  cost 

4.  The  generosity  of  the  W.K.  Kellogg  Foundation  which  supports  the 
remaining  portion  of  the  program 

The  new  QAP  focuses  on  sound  initiation  of  a  program,  identifying  and 
solving  problems  on  a  departmental  basis,  monitoring  the  process  of  problem- 
solving,  developing  a  sound  methodology  for  quality  assurance,  and  developing 
a  cost-effective,  centerwide  approach  to  formal  quality  assurance. 

The  program  was  developed  by  the  project  director,  who  was  responsible 
for  initiating  and  implementing  elements  of  the  program  and  methodologic  design 
in  conjunction  with  the  SLPMC  president,  a  physician  consultant,  the  board  of 
trustees  and  the  medical  director. 

The  project  director  initiated  communication  regarding  QAP  developments 
with  the  SLPMC  board  of  trustees  in  December,  1974,  which  led  to  five  formal 
presentations  before  the  board.    An  introductory  presentation  was  made  to  the 
board  president  about  the  specific  goals  and  program  elements  of  the  prospec- 
tive QAP.    Next,  the  project  director  made  presentations  to  a  special  subcom- 
mittee and  then  to  the  full  board  of  trustees  in  January,  1975.    Each  presentation 
built  on  previous  interaction,  proposing  new  objectives  and  agenda  items  to  be  dealt 
with  at  subsequent  meetings.    The  project  director  then  made  final  formal  presen- 
tations to  the  board  in  April,  1975,  proposing  approval  and  endorsement  of  the 
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QAP.    Following  that  presentation,  the  project  director  arranged  a  meeting 
between  the  president  of  the  board  and  members  of  the  Kellogg  Foundation. 
After  endorsement  of  the  program,  the  methodological  and  management  elements 
of  the  program  were  implemented. 

The  approach  adopted  by  the  project  director  for  initially  establishing 
the  QAP  was  very  formal,  insuring  that  implementation  of  the  program  acknowledged 
existing  managerial  and  supervisorial  relationships  and  that  broad  understanding 
of  program  progress  was  possible.    The  operations  components  of  the  QAP  were 
developed  concurrently  by  the  project  director.    Formal  procedures,  in  a  sequen- 
tial pattern,  were  used  to  establish  and  build  institutional  commitment  for 
the  program  by  allowing  the  board  of  trustees  to  closely  monitor  the  origin  of 
the  QAP.    As  the  project  developed,  several  practical  areas  required  careful 
investigation  and  study  prior  to  selection  of  a  mode  of  implementation.  These 
studies,  adopting  a  management  perspective  to  implementing  quality  assurance 
techniques,  played  a  key  role  in  the  initial  operations  of  the  program. 

SLPMC-QAP  documents  reviewed  describe  four  program  phases.    This  section 
will  outline  plans  for  each  phase  based  on  the  information  compiled  from  those 
documents.    The  section,  "Operational  Activities,"  will  outline  the  sequence 
of  operational  activities  documented  during  the  site  visit.    All  phases  can  be 
considered  mutually  exclusive,  independent  quality  assurance  programs;  however, 
each  phase  builds  toward  a  more  sophisticated  and  comprehensive  program. 

In  terms  of  developing  specific  program  plans,  SLPMC  has  emphasized  exten- 
sive documentation  of  future  plans,  reflecting  a  high  degree  of  specificity  in 
Phases  I  and  II,  now  well  underway,  and  less  detail  in  Phases  III  and  IV.  SLPMC's 
program  established  one  goal  for  each  phase,  with  a  number  of  corresponding  ob- 
jectives.   Each  of  the  four  program  phases  asks  a  question  about  medical  practice: 

Phase  I:     What  do  we  all  agree  can  be  done  better? 

Phase  II:    What  do  we  do  usually? 

Phase  III:  Do  we  meet  our  patient's  expectations? 

Phase  IV:    Are  new  quality  assurance  methods  effective  in  our  own  setting? 
Phase  I:    The  overall  first  year  goals  are: 

1.  To  develop  and  install  a  method  for  identifying  and  remedying  at 
least  25  significant  medical  care  problems  in  all  clinical  activities 
of  SLPMC 

2.  To  conduct  at  least  one  problem  identification  session  in  each  clini- 
cal department 

3.  To  integrate  the  method  in  routine  SLPMC  operations 

There  are  eight  specific  activities  in  the  Phase  I  of  the  quality  assurance 
program: 

1.  Departmental  meetings  are  held  during  which  physicians,  nurses  and  ad- 
ministrative staff  identify  one  specific  problem  for  review  and  assessment. 

2.  Department  personnel  develop  a  statement  of  the  identified  problem  and 
appoint  a  remedy  coordinator,  who,  along  with  the  QAP  staff,  is  respon- 
sible for  implementation  of  the  problem  solution. 
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3.  Appropriate  data  are  collected  for  identified  problem 

4.  A  suitable  data  base  for  the  problem  is  established 

5.  A  pretest  is  conducted  to  discover  the  extent  of  the  identified  problem 

6.  QAP  staff  meets  with  the  remedy  coordinator  to  discuss  proposed  remedy 
and  plans  for  implementation.    Remedy  coordinator  then  consults  with 
clinical  department  medical  staff. 

7.  Remedy  is  implemented  by  the  remedy  coordinator,  in  conjunction  with 
department  manager,  administrative  personnel,  and  QAP  staff  or  a 
combination  of  these 

8.  Remedy's  effectiveness  is  tested  with  a  reevaluation  procedure  that 
duplicates  pretest  method 

In  pretesting  problems  and  evaluating  remedies  in  Phase  I,  many  different 
methods  are  used,  dictated  by  specific  problems  identified. 

Phase  II:    The  overall  goal  is  to  gather  and  display  data  from  an  object- 
ive screening  of  the  universe  of  ambulatory  patient  care  experiences  to  facilitate 
internal  quality  assurance  activities  (started  in  Phase  I). 

SLPMC  documentation  notes  Phase  II  is  largely  scheduled  for  implementation 
during  the  second  program  year.    Nevertheless,  several  tasks  were  addressed 
according  to  the  implementation  plan  during  Phase  I.    While  Phase  I  concentrated 
on  subjective  identification  of  problems  by  professional  staff  members,  Phase  II 
is  designed  to  objectively  analyze  patient  care,  which  should  reveal  a  different 
problem  set. 

There  are  four  specific  activities  in  Phase  II: 

1.  Identification  and  prioritization  of  specific  outputs  of  Phase  II 
activities 

2.  Implementation  of  routine  outputs  based  on  the  computerized  Medical 
Service  Record  (MSR)  information 

3.  Establishment  of  parameters  for  specific  practice  profiles  with  in- 
formation generated  through  computerized  information  about  diagnoses, 
reasons  for  visit  and  procedures 

4.  Integration  of  outputs  into  Phase  I  activities 

Phase  III:    The  goal  is  to  develop  and  install  a  system  for  regular  patient 
feedback  about  the  care  they  expect  and  have  received  to  enable  appropriate  modi- 
fication of  patient  behavior  and  provider  performance  by: 

1.  Selecting  and  implementing  a  method  for  evaluating  and  quantifying 
patient  and  physician  expectations  of  medical  care 

2.  Assisting  in  the  development  and  evaluation  of  patient  and  provider 
education  programs  to  encourage  appropriate  expectations  and  modify 
inappropriate  expectations 

3.  Integrating  these  formalized  insights  into  Phase  I  activities 

Phase  IV:    The  goal  is  to  identify,  modify  and  integrate  additional  quality 
assessment  techniques  into  the  QAP  (Phases  I,  II  &  III)  by  performing  an  annual 
survey  of  new  external  developments  in  quality  assurance  such  as  public,  and  pri- 
vate regulatory  activities. 
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Phases  I  to  IV  describe  activities  which  gradually  build  a  more  sophisticated 
and  comprehensive  program.    Total  program  evaluation  must  await  implementation 
of  all  four  phases,  although  operational  evaluation  of  each  phase  is  regularly 
conducted.    Each  phase,  especially  Phase  I,  is  intended  to  function  as  an  indi- 
vidual quality  assessment  program  with  specific  goals  and  results. 


OPERATIONAL  ACTIVITIES 

The  full  implementation  of  Phase  I  and  initial  implementation  of  Phase  II 
were  completed  based  on  the  plans  outlined  in  the  preceding  section.    Most  opera- 
tional activities  matched  those  planned.    It  is  important  to  note  some  activities, 
although  not  formally  planned,  were  implemented  as  key  components  of  the  program 
(Point  2,  below).    This  seems  to  emphasize  the  importance  SLPMC  places  on  managing 
a  quality  assurance  approach,  rather  than  just  implementing  activities. 

Implementation  activities  of  Phase  I  actually  occurred  as  follows: 

1.  QAP  staff  scheduled  14  clinical  department  meetings  to  identify  prob- 
lems. 

2.  Prior  to  conducting  departmental  meetings,  the  QAP  staff  met  with 
various  administrative  personnel,  including  patient  educator,  HMO 
advisor,  patient  counselor  and  business  manager  to  discover  if  they 
had  any  identifiable  problems  to  be  included  in  identification  ses- 
sions. 

3.  Fourteen  departmental  meetings  were  held  involving  an  orientation  by 
the  medical  director,  asking  physicians  to  identify  significant  problems 
in  medical  care  which  can  be  remedied.    Over  200  problems  were  identi- 
fied. 

4.  After  identification  of  several  problems,  department  personnel  were 
required  to  select  only  one  problem  for  pretest.  Meetings  included 
the  QAP  staff,  manager  of  the  department,  head  nurse  of  department, 
medical  director,  and  departmental  physicians. 

5.  Physician  remedy  coordinators  were  selected  for  each  department. 

6.  QAP  staff  subsequently  met  with  remedy  coordinator  to  refine  the 
statement  of  problem  and  to  develop  method  to  pre-  and  post-test 
problem  and  design  remedies. 

7.  Remedy  coordinator  reported  back  to  department  to  ascertain  agreement 
on  statement  of  problem  and  remedy  method. 

8.  QAP  staff  pretested  the  problem  to  determine  problem  extent  and  fre- 
quency.   Nine  pretests  were  completed.    The  development  of  method 
involves  identifying  the  data  elements  necessary  to  quantify  the  prob- 
lem, such  as  the  development  of  explicit  screening  criteria  by 
departmental  physicians  for  problem  measurement,  examining  a  statis- 
tical minimum  sample  of  medical  records,  or  some  other  appropriate 
data  source  to  collect  relevant  data  for  each  problem.    No  one 
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systematic  methodology  is  applied  for  all  problems  identified,  or 

any  specific  format  to  construct  departmental  consensus  on  an  approach 

for  examining  and  solving  problems. 

9.    QAP  presented  pretest  results  to  remedy  coordinator,  whereupon  the 
remedy  was  implemented.    Ten  remedies  were  implemented.    QAP  staff 
indicated  different  individuals  may  participate  in  implementing  solu- 
tions, but  the  remedy  coordinator  has  direct  responsibility  for  imple- 
mentation. 

10.  Remedies  were  reassessed  after  an  appropriate  amount  of  time  had 
elapsed  (usually  three  to  six  months),  employing  the  same  method  used 
in  the  pretest. 

11.  Results  of  successful  problem  resolutions  were  then  publicized  by 
the  QAP  staff. 

12.  All  effective  remedies  are  periodically  reassessed  to  assure  that  the 
remedy  continues  to  be  effective. 

In  terms  of  initial  implementation  of  Phase  II,  QAP  staff  has  identified 
four  data  elements  to  be  systematically  collected  from  the  MSR  encounter  for 
development  of  a  routine  data  base  to  describe  or  develop  profiles  of  ambula- 
tory care  at  SLPMC: 

1 .  Reason  for  visit 

2.  Procedures 

3.  Diagnosis 

4.  Cost 

This  process  is  scheduled  for  a  later  date.  Other  tasks  were  addressed  during 
this  period,  including: 

1.  The  establishment  of  access  to  the  patient  care  data  base 

2.  The  identification  of  the  most  appropriate  diagnosis  and  problem  or 
reason-for-visit  code(s) 

3.  The  performance  of  selected  in-depth  analyses  using  the  patient  care 
data  base 

4.  The  analysis  of  patient  numbering  systems  at  the  primary  hospital 

5.  The  establishment  of  early  liaison  with  the  local  PSRO 


ADDITIONAL  COMMENTS 

Several  unique  characteristics  of  SLPMC  were  identified  for  study  staff 
during  the  site  visit.    These  unique  items  were  noted  in  relation  to  their 
importance  to  development  and  implementation  of  formal  quality  assessment 
activities : 

1.  Full  support  of  administration  and  medical  staff 

2.  Positive  attitude  of  physicians  toward  formal  quality  assurance  pro- 
cess 
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3.  Personal  commitment  of  top  administrative  and  medical  staff 

4.  Direct  role  of  project  director  in  all  phases  of  program  development 
and  implementation 
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SECTION  III 


DELIVERY     SYSTEM  CHARACTERISTICS 


SLPMC  has  78  physicians  providing  mul tispecialty  ambulatory  and  inpatient 
services  through  17  departments.    There  are  30  services  within  those  departments, 
as  follows: 


Specialty 

Allergy 

Cardiology 

Dermatology 

ENT 

Eye 

Family  Practice 
Industrial  Medicine 
Internal  Medicine 
Neurology 


No. 
Physicians 

4 
4 

3 
3 
3 
9 
2 

14-1/2 
1 


Specialty 

OB-GYN 
Oncology 
Pediatrics 
Surgery,  General 
Surgery, 
Surgery, 
Urology 
X-Ray 


Orthopedic 
Thoracic 


No. 
Physicians 

6 
4 

8-1/2 

5 

5 

1-1/2 

2 

2 


Other  services  include  the  provision  of  Patient  Education  Programs,  Family 
Counseling  Programs,  Physician  Education  Programs  and  Nurse  and  Paramedic  Educa- 
tion Programs.    Allied  services  of  dietitian,  physical  therapist,  orthoptist  and 
audiologist  are  also  provided. 

SLPMC  serves  a  total  patient  population  of  approximately  210,000  people, 
primarily  middle  and  upper  social  class.    In  1975,  377,000  patient  visits  took 
place.    There  were  23,504  new  patients  last  year.    The  majority  of  the  popula- 
tion lives  within  a  25-mile  radius  of  the  medical  center: 


76%  -  from  suburban  Minneapolis 
17%  -  from  city  of  Minneapolis 
7%  -  from  outside  metropolitan  area 


Between  January  1  and  August  24,  1975,  total  charges  for  209,945  patient 
visits  amounted  to  $6,286,814. 

Although  predominantly  a  fee-for-service  practice,  a  SLPMC-sponsored  closed 
panel  prepaid  plan  accounts  for  approximately  12%  of  patient  activity. 

The  organizational  structure  consists  of  a  business  administrator  and  a 
chief  medical  executive-president  elected  to  a  one-year  term  by  the  Board  of 
Trustees.  Members  of  the  board  are  selected  for  a  three-year  term  by  owners 
of  the  Beneficial  Interest  and  Trust.  The  board  convenes  twice  a  month.  The 
president  and  the  chief  of  the  medical  staff  are  responsible  for  the  medical 
business  of  the  SLPMC.  Clinic  operations  are  decided  through  statements  of 
policies  for  nonphysician  personnel  and  members  of  the  clinic  medical  staff. 
The  clinic  medical  staff  discusses  policies  at  weekly  department  meetings. 

Of  78  physicians,  one  is  a  regular  consultant  in  pathology;  one  more  phy- 
sician is  full  time,  but  totally  occupied  by  clinic  research.    The  support 
staff  is  illustrated  in  Figure  1. 
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Figure  1 


General  Support  Staff  Positions  No.  Personnel 

Administrator  1 

Administrative  Assistant  1 

Registered  Nurse  19 

Licensed  Practical  Nurse  33 

Physician  Assistant  8 

Nurse  Practitioner  3  (FTE) 

Office  Medical  Assistant  10 

Registered  Laboratory  Technician  10 

Laboratory  Assistant  14 

X-Ray  Technician  7 

X-Ray  Secretary  1 

Secretary  6 

Receptionist  40 

Record  Room 

Registered  Medical  Records  Technician  1 

Clerk  32 

Medical  Secretary  27 

Business  Office 

Business  Manager  1 

Cashier  2 

Insurance  Clerk  7 

Billing  Clerk                                 •  8 

Credit  Clerk  7 

Accountant  1 

Accounts  Payable  Clerk  2 

Data  Processing 

Supervisor  1 

Keypunch  and  Verifying  Operators  6 

Computer  Operator  1 

Ancillary  Departments 

Physical  Therapist  2 

Audiologist  1 

Electroencephalogram  Technician  1 

Electrocardiogram  Technician  4 

Psychiatric  Social  Worker  2 

Clinical  Psychologist  1 
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Chronological  medical  records  are  filed  manually  on  a  Remington-Rand  open 
shelf  rack  using  prenumbered,  precolor  coded  terminal  digit  chart  folders  and 
Soundex  Indexing  consoles. 

The  computer  system  currently  in  use  at  SLPMC  is  a  Univac  90/30  system 
configuration  with  central  processor,  an  operator's  CRT  display  console,  an 
integrated  card  reader  (500  CPM),  an  optimal  integrated  card  punch  (75-160  CPM), 
an  integrated  printer  (500  CPM)  and  two  integrated  disk  drivers  with  a  capacity 
of  57.8  MB.    Memory  sizes  range  from  a  minimum  of  32,768  bytes  to  a  maximum  of 
262,144  bytes  of  data. 

Characteristics  of  the  computer  are: 

1 .  131  K  core  (memory) 

2.  No  "canned"  programs  for  special  study 

3.  "ALL"  languages  can  be  used  for  programming 

4.  Temporary  disk  storage  is  available 

SLPMC  staff  have  several  service  affiliations: 

-  Conduct  general  surgery  and  0B-GYN  residency  programs  in  conjunction 
with  the  University  of  Minnesota. 

-  Conduct  a  Family  Practice  residency  program  in  conjunction  with 
Methodist  Hospital  and  the  University  of  Minnesota. 

-  Share  in  the  rotation  of  fellows  with  the  University  and  the  Hennepin 
County  General  Hospital  in  areas  of  Oncology,  Gastroenterology  and 
Pulmonary  Medicine. 

-  Teach  at  Hennepin  County  General  Hospital,  Minneapolis  VA  Hospital  and 
St.  Paul  Ramsey  Hospital. 

As  noted  above,  St.  Louis  Park  Medical  Center's  quality  assurance  program 
depends  on  a  high  degree  of  interaction  between  the  clinical  departments,  adminis- 
trative personnel  and  the  quality  assurance  staff.    The  method  of  assessments 
is  built  on  that  strong  interaction,  which  will  determine  the  continued  success 
of  the  program. 
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SECTION  IV 

HCMS     STUDY  INFORMATION 


SLPMC  agreed  to  participate  in  this  study  by  telephone  on  October  21,  1975, 
and  by  follow-up  correspondence,  November  25,  1975.    Program  documentation  re- 
ceived and  compiled  included: 

1.  Fact  sheet  on  the  delivery  system  of  SLPMC 

2.  Medical  Record 

3.  Questionnaire  filed  by  SLPMC  with  AMA  AMC  headquarters  for  accredita- 
tion 

4.  MSR  forms  utilized  by  physicians  during  patient  encounters 

5.  A  detailed  description  of  the  QAP  incorporated  the  following: 

a.  Initial  quality  assurance  plan  document  (dated  July  1,  1975) 

b.  QAP  goals  and  objectives 

c.  QAP  schedule  and  outline 

d.  Outline  of  QAP  presentation  (slide  presentation) 

e.  List  of  advisors  and  their  roles  in  the  QAP 

f.  Quality  assurance  questionnaire  given  to  entire  physician  staff 

g.  Quality  assurance  meeting  schedule  and  format 

h.  Flow  chart  indicating  specific  steps  and  activities  involved  in 
the  assessment  of  quality 

i.  List  of  problems  generated  by  each  department  to  be  focused  on 
by  the  QAP 

j.    List  of  administrative  problems  selected  for  resolution  by  SLPMC 

administration  and  QAP  staff 
k.    Status  sheet 

This  documentation  outlines  the  goals,  objectives,  general  program  elements, 
time  schedules  and  resource  commitments  for  the  SLPMC  Quality  Assurance  Program 
over  a  four-year  period.    The  Data  Collection  Instrument  (DCI)  and  Administrative 
Constraints  Questionnaire  (ACQ)  were  administered  during  the  site  visit,  January  26 
and  27,  1976. 

The  site  visit  resulted  in  120  minutes  of  taped  interviews.    Questions  were 
administered  to  the  following  SLPMC  personnel: 

1.  President  of  SLPMC 

2.  Medical  Director 

3.  Project  Director  of  QAP 

4.  Executive  Director  of  QAP 

5.  Biostatistician,  QAP 

6.  Assistant  Administrator,  SLPMC 

Topics  were: 

1.  Staffing  for  the  quality  assurance  program 

2.  Development  of  the  quality  assurance  program  with  the  board  of  trustees 

3.  Pros  and  cons  of  particular  quality  assurance  methods 
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4.  Specific  activities  of  quality  assurance  program  staff 

5.  Content  of  departmental  problem-identification  meetings 

6.  Unique  variable  of  SLPMC's  involvement  in  a  formal  quality  assurance 
program 

7.  Short  and  long-range  goals  of  quality  assurance  program 
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HARVARD  COMMUNITY  HEALTH  PLAN 
Boston,  Massachusetts 


SECTION  I 
INTRODUCTION 


Harvard  Community  Health  Plan  (HCHP),  in  operation  since  1969,  is  a  large 
prepaid  group  practice  consisting  of  two  health  centers  licensed  by  the  Massa- 
chusetts Department  of  Public  Health  (MDPH):    the  Kenmore  Center  with  an  out- 
reach program  in  Roxbury,  and  the  Cambridge  Center  which  opened  in  early  1975. 
The  Quality  Assurance  Program  (QAP)  of  HCHP  began  officially  in  October,  1974, 
and  selects  problems  or  diagnostic  conditions  for  study.    Usually,  standards 
are  developed  for  the  management  of  the  problem  or  diagnosis,  and  information 
is  collected  to  assess  delivery  system  performance  vis-a-vis  the  review  topic. 
The  quality  assurance  activities  at  the  Kenmore  Center  are  the  subject  of  this 
document. 

The  basic  data  set  for  this  program  is  the  computerized  medical  record 
known  as  COSTAR  -  The  Computer  Stored  Ambulatory  Record.    COSTAR  is  the  re- 
sult of  a  joint  effort  of  HCHP  and  the  Laboratory  of  Computer  Science  (LSC) 
at  Massachusetts  General  Hospital.    It  has  taken  six  years  to  develop  and 
implement  and  expands  upon  the  capabilities  of  the  traditional  medical  record. 
HCHP  has  a  joint  contract  with  LCS  for  the  maintenance  of  COSTAR. 

A  board  of  directors  oversees  the  policy  direction  of  HCHP  in  conjunction 
with  an  executive  committee.    The  two  bodies  interact  on  important  matters 
with  the  plan's  key  administrative  staff,  the  leadership  of  the  physician's 
group  and  the  Consumer  Advisory  Council.    The  medical  director  and  two 
associate  medical  directors  (who  each  serves  as  director  of  one  of  the  health 
centers)  have  general  responsibility  for  overseeing  care  provided  by  Plan 
staff.    The  Quality  Assurance  Committee  coordinates  quality  assurance  activi- 
ties.   Its  chairman  during  the  first  two  years  has  been  the  medical  director. 

There  are  60  physicians  employed  by  HCHP,  34  of  whom  practices  at  the 
Kenmore  Center.    Virtually  all  members  of  the  clinical  staff  hold  joint 
appointments  at  the  Harvard  Medical  School,  and  85  to  90%  of  the  staff  are 
board  certified. 

Harvard  Community  Health  Plan  was  asked  to  participate  in  the  study 
group  for  the  following  reasons: 

1.  It  is  a  large  group  practice 

2.  Numerous  quality  assurance  activities  have  been  implemented 
in  this  setting 

3.  The  operation  is  facilitated  by  a  very  sophisticated  computer 
system 
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SECTION  II 

QUALITY     ASSURANCE  PROGRAM 


This  section  will  describe  HCHP  quality  assurance  activities  in  three 
areas: 

1.  Background  of  program  development  and  the  initial  developmental 
period 

2.  Elements  of  quality  assurance  program  described  in  HCHP  documents 

3.  Operational  quality  assurance  activities 

Information  used  in  this  section  came  from  three  sources: 

1.  Documentation  outlining  some  program  plans  and 
results  of  operational  activities 

2.  Quality  Assurance  Committee  minutes 

3.  Investigators'  notes  from  interviews 


BACKGROUND 


The  documentation  and  interviews  revealed  seven  factors  which  brought 
about  the  development  of  the  HCHP  quality  assurance  program: 


1.  Interest  and  support  given  by  the  Dean  of  Harvard  Medical  School 

2.  Interest,  support  and  commitment  expressed  by  the  physicians  and 
nurses  on  the  staff  of  HCHP 

3.  The  fact  that  physicians  were  able  to  direct  their  feelings  about 
quality  review  and  quality  assurance  toward  the  total  goals  and  ob- 
jectives of  the  program  and  not  at  specific  individual  error 

4.  The  availability  of  a  diverse  patient  population  of  HCHP 

5.  External  pressures  presented  by  PSRO  and  HMO  laws 

6.  Need  for  follow-up  to  some  of  the  quality  assurance  work  already 
done  at  HCHP 

7.  COSTAR  (automated  medical  records  system) 


FORMALLY  PLANNED  ACTIVITIES 

The  first  organized  effort  in  quality  assurance  began  in  1972-1973,  after 
an  extensive  literature  review.    The  Department  of  Internal  Medicine,  through 
a  contract  from  the  Community  Health  Services  Division,  implemented  three  acti- 
vity areas: 

1.  Quarterly  tabulations  of  diagnosis  and  medications  prescribed  by  indi- 
vidual physicians  in  Internal  Medicine 

2.  Frequency  distributions  of  utilization  by  individual  members 

3.  Specific  quality  of  care  problems  audited  according  to  developed  ex- 
plicit process  criteria 
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In  each  aspect  of  the  study,  some  useful  information  was  obtained  and 
certain  methodological  problems  were  identified.    The  documentation  noted 
the  study  focused  primarily  on  three  sequential  steps  which  may  be  part  of 
any  quality  assurance  program:    (a)  selecting  tracers,  (b)  establishing 
criteria,  and  (c)  auditing  performance.    However,  study  results  indicated  to 
HCHP  personnel  that  measuring  improved  quality  and  assessing  criteria  required 
more  emphasis.    Consequently,  during  late  1973  and  early  1974,  plans  were  made 
to  establish  a  formal  quality  assurance  program,  centered  around  a  Quality 
Assurance  Committee.    In  the  fall  of  1974  the  HCHP  medical  director  moved  to 
establish  a  formal  method  and  an  administrative  program.    After  discussions  and 
deliberations  among  HCHP  provider  staff,  the  HCHP  Quality  Assurance  Committee 
(QAC)  was  formed. 

Documentation  indicates  the  Quality  Assurance  Committee  consists  of  11 
members  including  physicians,  RNs  and  administrative  staff.    Their  function 
is  to  insure  performance  of  the  QAP  in  the  major  clinical  specialties 
represented.    Each  specialty  and  department  is  responsible  for  developing  its 
own  priorities  for  quality  assurance  activities  and  studies.    Meetings  are 
held  every  two  weeks  between  7:30  and  8:30  a.m.    Members  from  the  varied 
clinical  specialties  meet  to  discuss  new  quality  assurance  activities  and 
review  current  actions.    The  chairman  of  the  committee  selects  an  agenda  for 
each  meeting  and  has  the  minutes  reproduced  and  distributed  to  the  committee 
members . 

The  formal  QAP  is  only  one  aspect  of  HCHP  quality  assurance  activities, 
which  also  include: 

1.  Established  standards  for  selection  of  MDs 

2.  High  degree  of  staff  interest  in  quality  assurance 

3.  Use  of  the  POMR  format  and  common  records,  and 

4.  Dynamics  of  group  practice  mode 

According  to  the  proposal  submitted  to  the  Public  Health  Service,  the 
goals  and  objectives  of  the  formal  QAP  are  as  follows: 

1 .  To  document  impact  on  the  provider  and  system  behavior  of  con- 
current feedback  to  the  provider  regarding  individual  cases  in 
which  inadequate  compliance  with  agreed  upon  standards  is  detected 

2.  To  document  the  usefulness  of  a  computerized  ambulatory  medical 
record  system  in  monitoring  compliance  with  the  requirements  for 
good  care  established  by  the  QAP 

3.  To  develop  as  an  integral  component  of  the  health  care  delivery 
system  a  QAP  which  uses  both  process  and  outcome  methodology  and 
meets  the  formal  requirements  of  P.L.  93-222  (HMO  Act  of  1973) 
for  a  QAP 

4.  To  report  the  experience  derived  from  the  QAP  to  assist  other  HMOs 
as  well  as  PSROs  involved  in  quality  assessment  of  ambulatory  care, 
in  preparing  programs  suited  to  their  own  practice  setting 
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HCHP  and  LCS  have  developed  a  QAP  with  one  important  and  unique  charac- 
teristic, an  extensive  and  automated  data  system.    The  QAP  revolves  around  a 
four-step  process,  involving  data  collection  and  audit  support  functions  per- 
formed by  the  computerized  data  system  (COSTAR) .    These  four  steps  are  as 
fol lows: 

1.  Selection  of  tracers 

2.  Selection  of  criteria  and  standards  for  these  tracers 

3.  Audit  of  performance  based  on  computerized  criteria  and 
computerized  audit  functions 

4.  Feedback  from  the  QAC  regarding  corrective  action  and  improvement 
of  performance 

The  data  sources  for  quality  assurance  information  are  the  computerized 
encounter  form  (medical  record)  and  the  lab  and  X-ray  entries.    Medical  in- 
formation is  captured  as  a  provider  marks  a  precoded  entry  form,  although  he 
may  use  additional  modifiers  or  free  text.    The  information  is  entered  at  a 
computer  terminal  (CRT)  within  24  hours  by  clerical  personnel.  Information 
relating  to  the  encounter  is  available  on  demand  through  CRT-display; 
alternatively,  computer-generated  printouts  are  prepared  through  the  medical 
record  room  for  routinely  scheduled  patients. 

The  COSTAR  medical  information  system  supports  all  four  stages  of  the 
quality  assurance  program,  but  the  documentation  did  not  describe  the  support 
in  detail.    The  criteria  selected  are  all  in  computer  retrievable  form. 
Each  step  of  the  four-step  process  is  described  below: 


Step  One  -  Selection  of  Tracers 

Tracers  were  selected  on  the  basis  of  the  five  criteria  defined  by 
Kessner*  and  one  additional  criterion  developed  by  HCHP.    This  sixth  criterion 
deals  with  relating  the  significant  expenditure  of  time  and  money  by  the 
health  care  system  for  chosen  tracer  conditions.    The  tracers  chosen  for 
study  thus  far  have  been:    (a)  hypertension,  (b)  urinary  tract  infections, 
(c)  sore  throats,  (d)  diabetes  mellitus,  (e)  otitis  media,  (f)  serology  (for 
syphilis,  etc.),  (g)  pediatric  well  child    care,  (h)  prenatal  care, 
(i)  colorectal  cancer  and  (j)  pap  smear. 

Step  Two  -  Criteria  and  Standards  Development 

Each  department  develops  its  own  criteria  for  the  tracer  conditions.  The 
documentation  notes  the  usual  approach  is  to  develop  criteria  by  compromising 
between  ideal  care  based  on  expert  medical  opinion  and  a  clearly  unacceptable 
care  pattern.    Computerized  protocols  represent  boundary  conditions  of  accep- 
table care.    Since  the  criteria  are  computerized,  the  computer  establishes 
variance  based  on  explicit  and  exact  standards. 

Criteria  are  updated,  as  needed,  after  review  and  assessment  at  a  depart- 
mental level.    Departments  reevaluate  and  establish  new  performance  levels  based 


*Kessner,  David  M.,  Kalk,  Carolyn,  Singer,  James:    Assessing  health  quality  - 
the  case  for  tracers,  New  Engl  J  Med  288-189-193,1973. 
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on  existing  criteria  and  standards.    The  departmental  updates  and  reformulations 
are  accomplished  through  discussions,  consensus  and  use  of  initial  assessment 
data. 


Step  Three  -  Audit  of  Performance  Based  on  Computerized  Criteria  and  Computerized 
Audit  Functions 

This  step  places  emphasis  on  concurrent  feedback  (i.e.,  deviations  from  pre- 
determined criteria  of  care  of  individual  patients  are  fed  back  to  physicians 
for  correction).    In  addition,  aggregate  data  are  prepared  periodically  for  analy- 
sis.   To  identify  provider  inadequacy,  all  encounters  outlining  the  tracer 
description  are  for  audit  purposes.    Every  case  having  the  particular  disease 
problem,  lab  test  or  medical  care  activity  is  monitored.    Performance  is  audited 
according  to  computerized  criteria.    The  review  of  audits,  however,  is  conducted 
by  the  QAC  and  by  the  person  receiving  the  audit. 


Step  Four  -  Feedback  from  the  Quality  Assurance  Committee  Regarding  Improvement 
and  Corrective  Action  of  Performance 

The  Quality  Assurance  Committee -and  department  personnel  review  initial 
monitoring  and  attempt  to  assess  whether  deviations  occurred  in  the  care  process 
or  the  recording  process.    Initial  monitoring  involves  identification  of  patients 
with  gaps  in  criteria  and  notification  of  the  individual  provider  of  the  defi- 
ciency.   Feedback  action  is  addressed  to  the  area  in  which  the  failure  occurred. 

Feedback  as  determined  by  the  QAC  is  based  on  two  broad  criteria: 

1.  Feedback  and  education  must  be  relevant  to  a  particular  situation 

2.  Feedback  and  education  must  be  timely  and  must  affect  this  situation 

There  is  no  formal  mechanism  for  enforcing  corrective  action.    Most  devia- 
tions are  "system  failures"  rather  than  examples  of  deficits  of  knowledge  about 
what  to  do;  for  example,  hypertensive  patients  who  are  not  told  to  return  for 
follow-up  or,  who  are  told  to  return,  but  fail  to  do  so.    The  aggregate  data  on 
performance  is  used  for  educational  programs.    After  the  computer  audit  flags  a 
variant  case,  the  QAC  reviews  the  deviation.    A  notification  and  questionnaire 
attached  to  the  specific  case  that  was  reviewed  may  be  sent  to  the  provider.  The 
provider  then  has  the  opportunity  to  disagree  with  the  standard  or  to  state 
why  it  is  inapplicable  in  specific  situations,  and  may  respond  to  the  audit 
through  discussion  with  the  department  chairman.    The  department  chairman  is 
responsible  for  deciding  future  action.    The  judgment  of  the  committee  is  not 
systematically  validated,  although  extensive  discussions  occur  prior  to  this 
final  review  and  corrective  action. 

The  computerized  system  enables  the  provider  to  be  notified  of  a  failure 
in  compliance  with  a  specific  standard  at  any  time  while  the  patient  is  under- 
going treatment.    Therefore,  the  provider  can  correct  the  error  brought  to  his 
attention. 
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OPERATIONAL  ACTIVITIES 


To  determine  the  present  operational  status  of  quality  assurance  activities 
at  HCHP,  minutes  from  Quality  Assurance  Committee  (QAC)  meetings  were  compiled 
and  analyzed  (October  10,  1974  to  January  22,  1976)  in  conjunction  with  specific 
project  information  abstracted  from  the  Public  Health  Service  Grant  dated 
September  11,  1975  (from  LCS  and  HCHP  regarding  a  computer-based  ambulatory 
QAP).    Study  staff  attempted  to  match  activities  by  listing  the  particular 
disease  or  condition  under  review,  and  sequentially  listing  operational  activi- 
ties relative  to  that  topic.    Figures  1  through  11  describe  in  detail  the  quality 
assurance  activities  that  have  taken  place  at  HCHP  according  to  the  QAC  minutes 
and  Public  Health  Service  Grant.    Activities  are  listed  chronologically  according 
to  source  document.    Because  specific  dates  were  not  listed  in  the  grant,  HCMS 
staff  estimated  dates  for  some  activities. 

Although  the  documentation  indicates  quality  assurance  activities  are  co- 
ordinated by  the  QAC,  certain  activities  were  described  in  the  grant  that  were 
not  described  in  the  QAC  minutes  (see  Urinary  Tract  Infections  -  Figure  4). 
The  operational  quality  assurance  activities  are  more  varied  and  extensive 
than  the  documented  planned  activities  involving  the  audit  and  review  of  several 
specific  disease  conditions.    The  project  section  of  the  grant  also  failed  to 
describe  certain  activities  noted  in  the  QAC  minutes  (Encounter  Form,  Figure  1, 
Anemia,  Figure  2,  and  Breast  Lump  Surgery,  Figure  3).    The  activities  of  the 
remaining  topics  were  described  in  both  document  sources  (see  Serology,  Figure  5, 
Hypertension,  Figure  6,  Sore  Throat,  Beta  Hemolytic  Strep,  Figure  7,  Prenatal  Care, 
Figure  8,  Pediatric  Well  Child,  Figure  9,  Colorectal  Cancer  -  Rectal  Exam,  Figure 
10,  and  Pap  Smears,  Figure  11). 

The  responsibilities  of  the  QAC  are  as  follows: 

1.  To  coordinate  quality  assurance  activities 

2.  To  assure  that  quality  assurance  activities  are  taking  place 

3.  To  assure  that  adequate  follow-up  is  performed  where  problems 
are  found 

The  individual  specialties  have  responsibility  for  initiating  studies  and 
determining  appropriate  priorities.    The  process  by  which  all  of  this  happens 
appears  to  be  informal,  without  specific  guidelines. 

In  numerous  instances  (specifically  for  Anemia,  Figure  2,  Breast  Lump 
Surgery,  Figure  3,  Hypertension,  Figure  6,  Beta  Hemolytic  Strep,  Figure  7) 
activities  are  initiated,  including  protocol  development,  and  staff  inter- 
action occurs;  then  the  activity  disappears  from  documentation.  The  QAC  is 
an  active  committee  and  did  not  set  its  time  schedule  or  update  reports  on 
each  project  to  fit  the  review  team's  schedule.  Therefore,  it  was  difficult 
to  discern  whether  the  specialty  or  representative  or  QAC  did  not  pursue  a 
particular  topic,  or  whether  activities  were  continued,  but  were  not  reported. 

The  grant  documentation  summarizes  quality  assurance  activities  occuring 
for  specific  topics.    Topics  are  discussed  with  regard  to  Tracer  Choice  (the 
choice,  not  how  it  was  chosen);  Criteria  (the  actual  criteria,  not  how  they 
were  developed);  Audit  (type  of  study);  and  Improvement  (what  has  been  done 
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to  improve  conditions  and  where  activities  are  as  of  September,  1975).  No 
dates  are  given  except  for  a  specific  data  collection  period. 

The  QAC  minutes  are  detailed  regarding  dates  and  the  type  of  activity 
in  which  the  QAC  was  engaged  at  the  time,  but  information  on  activity  develop- 
ment up  to  this  point  is  lacking.    For  example,  from  what  department  did  the 
idea  for  Beta  Hemolytic  Strep  (Figure  7)  as  a  tracer  condition  originate? 

QAC  minutes  recorded  a  high  rate  of  attendance  at  the  7:30  a.m.  meetings, 
as  well  as  a  high  level  of  interaction  between  QAC  members  and  individuals 
from  different  clinical  specialties.    Each  representative  from  a  specialty 
addressed  quality  assurance  issues  pertaining  to  his  field  and  took  part  in 
determining  quality  assurance  issues  outside  the  specialty. 

The  specific  actions  taken  by  the  QAC  varied  somewhat  by  topic.  Some 
general  activities  are  listed: 

1.  Revision  of  topics  for  review 

2.  Revision  of  methodology  (essentially  data  collection  and  analysis) 

3.  Review  and  approval  of  topic  criteria  and  protocols 

4.  Discussion  of  implementation  of  study  for  topic 

5.  Recommendations  for  departmental  action 

6.  Analysis  and  discussion  of  data  collected  with  regard  to  each  topic 

7.  Discussion  of  reviews  conducted  within  the  context  of  each  clinical 
specialty 

The  HCHP  quality  assurance  effort,  as  documented,  reflects  a  high  frequency 
of  complex  interactions  between  a  number  of  HCHP  departments  and  physicians. 
The  following  figures  reflect  this  and  provide  specific  information  on  the 
types  of  issues  the  QAC  has  addressed  over  an  extended  period. 
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Figure  1 
QUALITY  ASSURANCE  ACTIVITIES 
IMPLICIT  CHART  REVIEW.  INTERNAL  MEDICINE 


SOURCE 

DATE        DOCUMENT  ACTIVITY 


10/10/74     Minutes     Chart  audit  of  ten  specific  characteristics  of  IM  for 

improvement  of  form 

11/21/74     Minutes     Medical  record  system  modified  to  contain  information 

required  for  F-U,  e.g.,  flow  charts,  date  of  visit,  etc. 

9/4/75      Minutes     Proposal  presented  to  QAC  by  IM  for  conducting  implicit 

chart  review 

10/2/75      Minutes     Issues  of  implicit  chart  review  raised  by  QAC,  e.g.,  (1) 

What  encounter  forms  to  focus  on;  2  to  12  months  old; 
(2)  physician-nurse  team 

10/30/75     Minutes     Proposal  brought  before  QAC  again;  discussion  and  endorse- 
ment of  issues  by  QAC 

IM  =  Internal  Medicine 
F-U  =  Follow-up 

QAC  =  Quality  Assurance  Committee 


Figure  2 
QUALITY  ASSURANCE  ACTIVITY 
ANEMIA 


SOURCE 

DATE  DOCUMENT  ACTIVITY 


10/24/74     Minutes     Dr.   to  check  with  Pediatrics  Department  regarding 

flags  for  abnormal  lab  procedures.    Dr.    to  go  back 

to  IM  to  obtain  acceptable  limits  for  study.  Beginning 
to  develop  detailed  specifics.    Will  be  reviewed  by 
Dr.  and  Dr.  of  IM. 
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Figure  3 
QUALITY  ASSURANCE  ACTIVITIES 
BREAST  LUMP  SURGERY 


DATE 


SOURCE 
DOCUMENT 


ACTIVITY 


10/10/74     Minutes  Dr. 


10/24/74  Minutes 


 explained  to  QAC  steps  taken  by  Surgery  Depart- 
ment to  compare  quantity  of  visits  by  surgery,  length  of 
stay  and  complication,  rates  for  hospital  patients,  reac- 
tions to  care  by  patients  from  questionnaire,  monitoring 
of  feedback  from  referring  physician  regarding  impression 
of  performance  at  Surgery  Department. 

To  get  50  records  (sample)  of  patients  with  surgery  (breast 
mass)  to  examine  care  and  F-U  records  chosen  from  10/1/73  - 
3/31/74. 


11/7/74      Minutes  Dr. 


and  Dr. 


looking  at  management  of  50  pa- 


tients with  diagnosis  "breast  lump"  and  considering 
approach  to  monitoring. 

12/5/74       Minutes     Notes  prepared  on  breast  lump  issue  distributed  to  QAC 

by  Dr.   . 

12/17/74     Minutes     Preliminary  results  of  review  of  45  charts  with  "breast 

mass"  presented  to  QAC  by  Dr.   .    Significant  finding 

was  insufficient  data.    Guidelines  for  minimum  criteria 
of  data  should  be  established  by  surgery  group.    This  will 
be  addressed  in  greater  detail  later. 


F-U  =  Follow-up 

QAC  =  Quality  Assurance  Committee 
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Figure  4 
QUALITY  ASSURANCE  ACTIVITIES 
URINARY  TRACT  INFECTION 


DATE 


SOURCE 
DOCUMENT 


ACTIVITY 


Grant 
Grant 

Grant 

Grant 


9/75 


Grant 


UTI  chosen  as  tracer  at  2.%  of  adult  visits. 

Protocols  established  for  women  16  years  and  over  with 
positive  urine  culture. 

A  preliminary  run  of  50  patients  with  positive  cultures 
performed;  33  found  without  adequate  F-U. 

Wished  to  know  if  failure  to  F-U  results  in  further  return 
visits  to  HCHP  with  symptom  infections.    Therefore,  did 
preliminary  retrospective  study  on  all  patients  with  posi- 
tive cultures  who  were  members  of  HCHP  for  one  year  follow- 
ing culture.    Patients  divided  into  two  groups,  those 
without  adequate  F-U  and  those  without  adequate  F-U  although 
data  not  analyzed  in  detail  yet;  no  difference  in  two  groups, 


UTI  program  at  a  crucial 
lished  on  preliminary  ba 
(according  to  criteria) 
past  treatment  urine  cul 
data  on  patients  receivi 
gested  follow-up  culture 
in  their  subsequent  vari 
UTIs.    A  more  extensive 


point  at  this  time.    Have  estab- 
sis  that  performance  is  inadequate 
66%  of  time  in  the  sense  that  a 
ture  not  determined.    Analysis  of 
ng  culture  not  receiving  the  sug- 

f ailed  to  reveal  any  difference 
ability  relative  to  recurrent 
retrospective  study  is  planned. 


UTI  =  Urinary  tract  infection 
F-U  =  Follow-up 
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Figure  5 
QUALITY  ASSURANCE  ACTIVITIES 
SEROLOGY 


DATE 


SOURCE 
DOCUMENT 


ACTIVITY 


10/10/74 


1974 

1975  x 
(early) 


6/26/75  Minutes 


7/24/75  Minutes 


Mid-1975  Grant 


9/75 


Minutes     F-U  study  on  patients  with  positive  serology  indicated. 

Grant      Implicit  review  of  101  positive  serologies  (Kenmore  pa- 
tients) performed  by  one  member  of  Department  of  Medicine 
and  HCHP  liaison  V.D.  Public  Health  Nurse.  Automated 
findings: 

(1)  Records  judged  inadequate  in  45  patients  based 
on  automated  data 

(2)  Liaison  nurse  unaware  of  16  patients 

(3)  Unnecessary  and  costly  repetition  of  FTA-Abs. 
tests. 

(4)  F-U  exams  and  serologies  almost  uniformly  not 
obtained 

More  ideas  on  F-U  of  abnormal  serologies  presented  to  QAC. 
Protocol  will  be  prepared  to  allow  for  print-out  of  all 
patients  with  positive  serologies  and  to  flag  those  with- 
out F-U.    Dr.   will  handle  review  to  assess  against 

criteria . 

Proposal  for  F-U  of  patients  with  positive  serologies  pre- 
sented and  endorsed  by  QAC.    Flow  chart  for  monitoring 
protocol  circulated 

Concurrent  review  set  up  based  on  initial  study  (grant  notes 
above).    It  will  be  a  manual  review  because  cases  are  few 
and  (1)  for  each  positive  serology  in  record,  computer 
generates  timely  reminders  to  obtain  necessary  F-U  exams, 
(2)  computer  generates  copies  of  record  for  nurse  and  Medi- 
cal Department  member.    Manual  audit  includes: 

(1)  Proper  classification 

(2)  Indication  of  previous  treatment 

(3)  Plans  for  evaluation  and  treatment 

Grant      A  before  and  after  study  of  adherence  to  criteria  will  be 
performed.    Plan  to  do  cost  accounting  with  this.    Will  not 
be  able  to  show  number  of  cases  without  tertiary  syphilis 
prevented  by  actions.    May  be  able  to  demonstrate  a  reduc- 
tion in  unnecessary  costs;  however  improved  F-U  =  more 
costs. 


F-U  =  Follow-up 

QAC  =  Quality  Assurance  Committee 
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Figure  6 
QUALITY  ASSURANCE  ACTIVITIES 
HYPERTENSION 


DATE 


SOURCE 
DOCUMENT 


ACTIVITY 


10/10/74 
10/24/74 


12/5/74 


End  of 
1974 

4/1  - 
12/31/74 
(study 
period) 


Minutes 
Minutes 


Early  Grant       Hypertension  chosen  as  tracer  (through  Department  of  In- 

1974  ternal  Medicine). 

Mid  1974       Grant      To  evaluate  F-U  of  abnormal  blood  pressures,  retrospective 

audit  conducted  on  all  HCHP  members  who  had  diastolic  pres- 
sure of  100+  or  were  diagnosed  as  hypertensive  prior  to 
1974.    Result  was  that  1/3  of  population  diagnosed  as 
"severe"  (120+)  had  inadequate  F-U  and  57%  diagnosed  as 
"mild"  (100+)  had  inadequate  F-U. 

Development  of  protocols  to  monitor  hypertensive  patients 
initiated. 

Need  to  go  back  to  IM  and  clarify  specifics.    0B  Department 
to  perform  analysis  on  hypertensives  found  in  0B-GYN  Depart- 
ment.   Prototype  for  hypertension  reviewed  and  revised. 
Membership  requirements  for  study  changed.    Also  need  to 
identify  specialty  where  blood  pressure  recorded 

Minutes     Hypertension  proposal  for  monitoring  patients  distributed 
to  QAC. 

Grant      Standards  for  F-U  of  hypertensive  patient  established. 

Grant      Computer  scanned  patients  over  age  16  and  identified  327 
HCHP  members  with  elevated  diastolic  pressure.    112  of 
these  (34.2%)  had  no  F-U  within  6  months.    These  cases  put 
into  random  control  and  random  experimental  groups.    If  in 
control,  no  action  taken.    If  experimental,  notification 
sent  to  provider  by  QAC  re  justification.    Results  thus 
far  are  failure  to  keep  appointments  and  overlooked  or  less 
important  problems. 

End  of  Grant       F-U  records  of  patients  being  examined  at  3-month  intervals 

1974  throughout  1975.    At  end  of  year  (75?)  will  be  able  to 

determine  if  review  led  to  improvement.    Goal  is  limited 

and  short-term. 

1/9/75        Minutes     Early  experience  with  hypertension  protocols  reported.  F-U 

only  taken  place  for  past  month.    Beginning  to  be  fed  into 
computer. 


F-U  =  Follow-up  0B  =  Obstetrics 

IM    =  Internal  Medicine  GYN  =  Gynecology 

QAC  =  Quality  Assurance  Committee 
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Figure  7 
QUALITY  ASSURANCE  ACTIVITIES 
SORE  THROAT-BETA  HEMOLYTIC  STREP  (BHS) 


DATE 


SOURCE 
DOCUMENT 


ACTIVITY 


10/24/74  Minutes 


11/7/74  Minutes 


10/10/74     Minutes     F-U  study  on  patients  with  positive  cultures  for  BHS  taken. 

Late  Grant      QAC  felt  that  standard  of  care  =  every  patient  with  posi- 

1974  tive  culture  for  strep  should  be  treated  within  4  days 

after  culture. 

Protocol  details  needed  on  results  so  far.    Steps  taken  in 
cases  found  with  positive  cultures  without  record  of  treat- 
ment.   Letters  sent  to  providers  re  why  treatment  not  given 
(on  monthly  basis  10/1/73-10/1/74). 

Method  for  study  re  F-U  of  patients  with  positive  BHS  de- 
veloped.   Patients  without  treatment  flagged  by  memo  from 
QAC.    Memo  of  rationale  circulated  to  QAC.    Data  presented 
on  number  of  cultures,  percent  positive  each  month  and 
percent  of  these  not  treated. 

11/74  Grant      Computer-based  concurrent  review  initiated:    (1)  every  case 

with  positive  BHS  noted  automatically,  (2)  if  no  entry  of 
antibiotics,  provider  notified,  (3)  computer  continues  to 
monitor  until  situation  remedied. 

1/9/75        Minutes     Early  experience  with  strep  protocols  reported  to  QAC  by 

Dr.   .    Preliminary  data  reveal  greater  than  50% 

failure  to  treat,  actually  failure  to  record  information; 
20%  on  form,  not  in  system;  30%  not  treated. 

Results:    (1)  Provider  notified;  (2)  Notification  oriented 
to  specific  patient  care  situation,  record  included;  (3) 
Provider  encouraged  to  respond  to  12  (4-day)  notifications 
sent.    Of  these,  1/3  treated,  not  recorded;  1/3  completed 
too  late;  1/3  failure  of  lab,  or  patient-provider  communi- 
cation breakdown. 

Year  Grant      Retrospective  analysis  of  HCHP  experience  of  "sore  throats" 

ending  reported  in  5,680  cases  (all  specialties). 

3/1/75 


1974,  75  Grant 


3/20/75       Minutes     Throat  culture  monitoring  presented  to  QAC  by  Dr. 


Areas  of  potential  monitoring  identified.  Additional  in- 
formation about  strep  study  to  be  circulated. 


F-U  =  Follow-up 

QAC  =  Quality  Assurance  Committee 
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Figure  8 
QUALITY  ASSURANCE  ACTIVITIES 
PRENATAL  CARE 


SOURCE 

DATE        DOCUMENT  ACTIVITY 


11/21/74  Minutes 


12/12/74 


End  of 
1974 


Minutes 
Grant 


1/23/75  Minutes 


2/3/75 


Grant 


3/75 


Grant 


Standards  developed 
Staff  will  develop 
lating  to  pregnancy 
establish  way  of  co 
variables.    Form  wi 
Boston  Hospital  for 
comes  and  separate 
cussed.  Important 
departments  for  pro 
plored  by  OB-GYN. 


for  conduct  of  prenatal  care  by  OB-GYN, 
tolerance  ranges.    Outcome  measures  re- 

and  newborn  form  will  be  reviewed  to 
rrelating  process  measures  with  outcome 
11  be  reviewed  by  Dr.   and  Dr.   

Women  asked  to  supply  reports  of  out- 
HCHP  patients.    Role  of  internist  dis- 
to  establish  communication  between 
blems  with  pregnancy.    Ways  being  ex- 


Dr. 


asked  to  indicate  to  QAC  standards  being  estab- 


lished and  tolerance  ranges. 

OB  Department  has  developed  a  set  of  process  criteria 
which  identify  elements  to  be  incorporated  in  a  prenatal 
service  program  for  every  pregnant  woman  receiving  care  at 
HCHP. 

OB  approach  for  monitoring  discussed  with  QAC.  Description 
of  summary  operations,  protocols,  etc.,  circulated.    25  re- 
cords reviewed  to  measure  compliance  with  protocol.  Con- 
clusion -  all  items  in  protocol  carried  out.    Steps  (sug- 
gested by  QAC)  made  to  integrate  show  and  no-show  in  computer, 

Dr.    asked  to  develop  monitoring  through  computer  for 

small  group  in  Cambridge. 

Computer  program  to  monitor  compliance  with  criteria  de- 
veloped. Program  will  check  for  performance  of  elements 
(1)  within  6  weeks  of  1st  visit,  (2)  for  each  subsequent 
visit,  (3)  at  appropriate  times. 

At  each  prenatal  visit,  a  flow-chart  summarizing  patients' 
compliance  with  criteria  will  be  presented  to  the  provider 
with  attention  directed  at  any  deficiencies.    After  preg- 
nancy, extent  of  compliance  with  requirements  for  1st 
prenatal  visit,  frequency  and  timing  of  last  visit  to  be 
described.    An  index  which  summarizes  the  degree  of  com- 
pliance with  full  set  of  standards  will  be  used  to  describe 
each  pregnancy. 


98 


Figure  8  continued: 


SOURCE 

DATE        DOCUMENT  ACTIVITY 


4/17/75       Minutes     Dr.    reported  (to  QAC)  on  initial  results  of  QA  pro- 

gram for  prenatal  patients  (3/31/75).    Preliminary  study 
looked  at  performance,  etc.  -  87%  compliance. 

8-9/75?        Grant      Re:    Boston  Hospital  for  Women  study.    This  would  involve 

isolating  data  for  pregnancy  outcome  for  HCHP  members, 
analyzing  rates  of  complications,  conditions  and  pregnancy 
outcomes  in  comparison  with  non-HCHP  women  at  Boston  Hospi- 
tal for  Women  and  determine  extent  to  which  a  correlation 
exists  between  outcome  and  process.    For  each  individual 
pregnancy,  a  score  will  be  computed  using  process  and  out- 
come measures  and  a  correlation  between  these  scores  for 
entire  population  determined.    Possible  throughout  study 
to  monitor  compliance  with  process  criteria.    Results  of 
study  will  be  prepared  for  each  physician-nurse  team, 
shared  with  teams  and  chief  of  service  and  used  as  basis 
for  developing  strategy  should  significant  problem  areas 
emerge.    Data  will  be  broken  down  by  source  of  enrollment 
for  different  risk  groups. 

10/30/75     Minutes     F-U  report  on  QA  preliminary  study  reported  to  QAC  by 

Dr.   .    Higher  compliance  rate  than  preliminary  study. 

Dr.    reported  efforts  underway  for  Boston  Hospital 

for  Women  to  provide  information  re  outcomes  of  pregnancy. 
Computer-acceptable  form  will  be  prepared  by  physician 
each  delivery  (to  compare  process  with  outcome).    A  high 
degree  of  enthusiasm  by  QAC  re  (1)  high  OB  compliance  and 
(2)  future  QAC-QA  possibilities. 


OB    =  Obstetrics 

GYN  =  Gynecology 

QA    =  Quality  assurance 

QAC  =  Quality  Assurance  Committee 
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Figure  9 
QUALITY  ASSURANCE  ACTIVITIES 
PEDIATRIC  WELL-CHILD 


DATE 


SOURCE 
DOCUMENT 


ACTIVITY 


10/24/74  Minutes 


11/7/74  Minutes 


11/21/74  Minutes 


12/19/74  Minutes 


1/9/75 


1/23/75 
2/6/75 


Minutes 


Minutes 
Minutes 


3/20/75  Minutes 


Pediatrics  Department  developing  specifics  for  immunization 
requirements,  standards  of  care  for  patients  up  to  6  months, 
Pediatrics  Department wi  1 1  be  asked  to  develop  specific 
proposal  for  monitoring  of  care.    Special  attention  will  be 
given  to  patients  seen  and  not  treated  vs.  those  not  seen. 

Plan  prepared  for  monitoring  well-child  care  presented  to 
QAC  includes:    (1)  items  to  be  monitored,  (2)  time  inter- 
vals for  flagged  patients  not  measuring  up  to  standards, 
(3)  membership  requirement  of  target  population. 


Proposal  presented  to  QAC  -  "minimum 
of  care  in  well -child  area."  Dr. 
weeks  with  recommended  criteria  and  tolerance 


criteria  for  quality 
to  return  in  4 
ranges . 


12/12/74     Minutes  Dr. 


  asked  to  prepare  memo  identifying  reasons  Depart- 
ment of  Pediatrics  monitors,  method  of  standard 
establishment,  actual  standards,  and  tolerance  ranges. 

Extensive  thought  given  by  Pediatrics  Department  to  mini- 
mum criteria  proposal.    Department  to  review  recommendation 
next  month  and  come  back  with  final  proposal  indicating 
age  groups  to  be  surveyed,  selected  criteria  for  priority 
and  time  tolerances. 

Proposal  for  monitoring  well-child  care  discussed.  Proto- 
col will  be  implemented  with  3  purposes:    (1)  children  less 
than  6  years,  (2)  children  who  are  subjects  of 
Supplementation  Evaluation  Program  on  health  costs,  (3) 

Dr.   and  Dr.   working  to  develop  F-U  on  study  of 

two  years  ago  on  missed  appointments. 

Common  protocol  to  meet  needs  of  above  being  developed. 

Program  developed  by  Pediatrics  Department  to  accommodate 

above  needs  circulated  and  accepted  by  QAC.  Dr.   

asked  to  facilitate  activities  as  soon  as  possible. 

First  computer  run  of  pediatric  utilization  and  immuniza- 
tion surveillance  made.    Data  delivered.  Pediatrics 
Department  presented  draft  questionnaire  for  contacting 
low  utilizers  and  not-up-to-date  immunizations.  Construc- 
tive criticism  of  questions  and  methods  made  by  QAC  to 
go  back  to  department  and  be  redrafted. 
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Figure  9  continued 


DATE 


SOURCE 
DOCUMENT 


ACTIVITY 


Summary 
1974-75 


3/75' 


Mid-late 
1975? 


Grant      Members  of  Department  of  Pediatrics  analyzed  various  cri- 
teria.   Table  for  minimum  criteria  for  quality  of  care 
established  with  standards  and  minimum  content  necessary 
for  adequate  care.    All  information  computerized. 

Grant      Department  of  Pediatrics  has  prepared  set  of  priorities 
for  making  direct  contact  with  parents  whose  children 
haven't  met  recommended  criteria.    System  will  monitor 
compliance  of  individual  patients  with  recommended  criteria, 
Direct  contact  will  be  made  by  letter  to  parents,  beginning 
with  highest  priority  group. 

Grant       Degree  of  compliance  of  particular  practitioner  with  rec- 
ommended schedule  will  be  monitored  and  information  will 
be  made  available  to  chief  of  service  and  to  each  practi- 
tioner. 


1/22/76       Minutes     Dr.    mentioned  (to  QAC)  study  of  immunization  status 

of  children  in  Boston  schools.    Dr.    introduced  dis- 

cussion of  analysis  of  compliance  with  well-child  program  at 
Kenmore.    Dr.   and  Dr.    said  that  much  noncompli- 

ance is  technical  error  and  small  degree  of  variation  in 
compliance  between  employer  groups. 


QAC  =  Quality  Assurance  Committee 
F-U  =  Follow-up 
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Figure  10 
QUALITY  ASSURANCE  ACTIVITIES 
COLORECTAL  CANCER  RECTAL  EXAMS 


SOURCE 

DATE        DOCUMENT  ACTIVITY 


9/4/75        Minutes     Dr.    introduced  protocol  proposal  (to  QAC)  presented 

and  developed  by  IM  for  monitoring  routine  rectal  exams 
for  patients  over  age  40.    Discussions  re  having  a  reminder 
for  each  patient  without  stool  guaiac  test.    QAC  felt  bringing 
lack  of  tests  and  F-U  to  attention  of  providers  would  result 
in  less  nonperformance.    QAC  to  review  proposal  in  greater 

detail  and  forward  to  IM  for  discussion.    Dr.    to  take 

it  back  to  Surqery.    Also  to  go  back  to  OB  as  a  rectal  exam 
and  stool  guaiac  has  not  been  performed  routinely  on  women 
over  age  40.    First  3  months  of  aggregate  data  showed  non- 
performance. 


9/75  Grant       Complete  protocol  criteria  developed  -  e.g.,  a  digital  exam 

and  stool  guaiac  test  to  be  obtained  at  all  Initial  Health 
Assessments  (IHA)  and  Periodic  Health  Reviews  (PHR)  of 
patients  below  age  40.    This  has  emerged  from  study  of  IM  and 
QAC.    Audit  can  be  fully  automated  except  where  there  are 
explained  deviations.    Deviant  records  will  be  reviewed  by 
member  of  IM  group  for  explanations.    Through  use  of  remin- 
ders to  do  agreed  upon  screening  procedures  and  F-U  abnor- 
malities, hope  to  achieve  high  level  of  compliance.  Con- 
current review  will  take  place  via  before  and  after  study 
and  RCT  (Randomized  Clinical  Trial). 

9/75  Grant      Departments  of  OB  and  Surgery  are  preparing  program  that 

will  complement  the  IM  screening  program.    OB  wishes  to 
assure  that  women  over  40  years  will  have  pelvic  and  rec- 
tal exams  with  stool  guaiac.    Surgery  preparing  program  that 
will  determine  how  effectively  it  is  dealing  with  abnor- 
malities detected  in  screening  program.    These  are  all  in 
preliminary  stages. 


10/30/75     Minutes     Proposal  for  screening  endorsed  by  QAC.    Program  for  colo- 
rectal cancer  discussed  second  time.    IM  discussed  this  in 
detail.    Attention  directed  to  methodology  of  notification 
system  re  nonperformance,  F-U  of  barium  enema,  etc.;  more 
comments  requested  from  0B  and  Surgery  Departments. 


IM   =  Internal  Medicine 
0B    =  Obstetrics 

QAC  =  Quality  Assurance  Committee 
F-U  =  Follow-up 
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Figure  11 
QUALITY  ASSURANCE  ACTIVITIES 
PAP  SMEARS 


DATE 


SOURCE 
DOCUMENT 


ACTIVITY 


11/21/74 
12/19/74 


6/12/75 
6/26/75 


Minutes 
Minutes 


4/17/75  Minutes 


5/29/75  Minutes 


Minutes 
Minutes 


Mid  1975 


Grant 


9/75 


Grant 


Current  state  of  pap  tests  reported  to  QAC  re  QA  program 
in  OB. 

Physician  reported  on  project  (to  QAC)  in  his  own  practice 
re  abnormal  paps.    Asked  to  discuss  results  (e.g.,  found 
small  percentage  of  abnormal  pap  results  have  evidence  of 
a  repeat  pap)  to  determine  whether  paps  should  be  put  into 
a  broader  proposal  and  brought  back  to  QAC. 

Modifications  have  occurred  since  original  proposal  for  F-U 
of  patients  with  abnormal  paps.    Warrants  further  discus- 
sion with  OB.    Several  suggestions  and  modifications  made 
by  QAC.    Discussed  anxiety  of  patients  with  abnormal  paps. 
OB  will  review  waiting  time.    Proposal  will  be  reviewed 
again. 

Revision  proposal  submitted  for  pap  analysis  and  F-U  (to 
QAC).    Major  discussion  on  "Pap  Smear  -  QA  Protocol." 
Confusion  as  to  time  span  and  target  population.  Proto- 
col will  be  resubmitted. 


Pap  protocol  rewritten  to  reflect  QAC  suggestions 
reviewed  again  by  QAC. 


and 


Further  discussion  needed  regarding  women  who  haven't  had 
paps  or  have  gone  without  them  for  two  years.    Decided  QAC 
not  to  develop  policy  for  this.    Raised  issue  of  developing 
more  focused  efforts  on  patients  with  expected  higher  risks 
To  be  referred  to  IM  for  reformulation  and  recommendations. 

Pap  smear  project  to  assure  that  abnormal  paps  are  followed 
up  grew  out  of  observations  of  one  member  of  IM.  Criteria 
development  has  required  close  cooperation  of  members  of 
IM  and  OB  and  representatives  from  LCS.    Progress  delayed 
by  introduction  of  colposcopy  -  a  procedure  to  improve 
fol low-up. 

Details  of  scheduling  patients  for  procedure  still  to  be 
analyzed  so  that  precise  criteria  have  not  yet  been  chosen. 
Preliminary  criteria  exist.  Concurrent  review  will  be  con- 
ducted to  survey  all  records  of  women  with  abnormal  pap. 
Provider  will  be  notified  within  designated  time  frames. 
Printouts  of  questionable  records  will  be  sent  to  desig- 
nate members  of  IM  or  OB  for  final  decision. 
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Figure  11  continued: 


SOURCE 

DATE        DOCUMENT  ACTIVITY 


9/75?  Grant      To  document  improved  F-U,  two  types  of  studies  will  be  con- 

ducted; before  and  after  and  RCT.    The  RCT  control  group 
will  be  audited  on  schedule  but  corrective  action  will  be 
delayed.    Results  will  be  compared  to  determine  if  concur- 
rent review  resulted  in  better  compliance  with  criteria. 


pap  smear  =  Papanicolaou  test 

QA  =  Quality  assurance 

QAC  =  Quality  Assurance  Committee 

OB  =  Obstetrics 

LCS  =  Laboratory  of  Computer  Science 

RCT  =  Randomized  Clinical  Trial 
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SECTION  III 

DELIVERY     SYSTEM  CHARACTERISTICS 


HCHP  provides  mul ti special ty  ambulatory  services  in  outpatient  health  cen- 
ters and  inpatient  services  at  various  Harvard  teaching  hospitals  in  the  Boston 
area.    The  hospitals  serving  Kenmore  members  are  as  follows: 


Hospital 
Boston  Hospital  for  Women 

Beth  Israel  Hospital 

Childrens  Hospital  Medical  Center 

Glenside  Hospital 

Peter  Bent  Brigham  Hospital 

Massachusetts  Eye  and  Ear  Infirmary 


Presenting  Problem 

Maternity,  obstetrics  and  gynecologi- 
cal problems 

Adult  and  Psychiatric  Care  Problems 
Pediatric  Problems 
Psychiatric  Care  Problems 
Adult  Care  Problems 
ENT  Problems 


Services  provided  by  HCHP  consist  of  Internal  Medicine,  Pediatrics,  Obste- 
trics and  Gynecology,  Surgical  and  Medical  Specialties,  Visual  Services,  Mental 
Health  Services,  Oral  Surgery,  Ear,  Nose  and  Throat,  Radiology,  Lab  Services, 
Complete  Physical  Examinations,  Preventive  Dentistry  for  Children,  Immunizations, 
Allergy,  Dermatology,  Neurology,  and  Orthopedics. 


HCHP  employs  60  physicians,  34  at  the  Kenmore  Center.  The  distribution 
of  FTE  (full  time  equivalent)  staff  by  clinical  specialty  is  as  follows: 


Specialty  FTE 


Obstetrics  and  Gynecology  4.00 

Internal  Medicine  12.25 

Psychiatry  3.93 

Surgery  2.25 

Pediatrics  4.50 

Radiology  1.20 

Orthopedics  0.45 

Dermatology  0.45 


Other  services  include  counseling  for  acute  psychiatric  problems,  Home 
Health  Services,  Extended  and  Intermediate  Care  Services,  Emergency  Care  in  and 
out  of  the  service  area,  and  Corrective  Appliances. 

The  health  centers  operate  from  8:30  a.m.  to  5:00  p.m.  Monday  through 
Friday.    The  triage  area  is  open  with  a  limited  staff  weekdays  until  9:00  p.m. 
and  daily  on  weekends.    Monday  is  the  busiest  day  of  the  week.    Off-hour  care 
is  provided  by  HCHP  physicians  on  rotating  shifts. 
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PATIENT  POPULATION 


,The  HCHP-Kenmore  Center  has  approximately  550  daily  visits  by  a  patient 
population  that  is  diverse  and  generally  representative  of  the  Boston  area. 
During  FY  1976  (10/75  -  9/76),  the  average  membership  was  55,250  (35,250  - 
Kenmore  and  20,000  -  Cambridge)  most  of  whom  are  enrolled  through  employers' 
prepaid  group  coverage  plans.  The  outreach  program  (Mission  Hill  -  Parker  Hill 
area)  services  a  predominantly  low-income  population. 

For  the  year  ending  June  30,  1974,  the  average  number  of  encounters  per 
person  was  4.2.    The  average  number  of  physician  and  nurse  encounters  per  per- 
son was  2.04  and  0.89,  respectively.    Over  80%  of  the  HCHP  population  lives 
within  8  to  10  miles  of  the  health  centers.    The  facilities  are  easily  acces- 
sible by  car  and  public  transportation. 


PROVIDER  SELECTION  PROCESS 

A  new  member  selects  a  physician-nurse  team  by  scheduling  an  initial  health 
assessment  with  the  team  of  his  choice.    If  a  member  is  less  than  16  years  of 
age,  a  pediatrician  and  pediatric  nurse  practitioner  are  chosen.    If  a  member 
is  over  16  years  of  age,  a  specialist  in  Internal  Medicine  and  an  adult  nurse 
practitioner  are  chosen.    Orientation  meetings  are  held  to  assist  members  with 
team  selection. 


REVENUE  SOURCES 

The  plan  operates  through  contracts  with  Massachusetts  Blue  Cross  and 
several  established  commercial  insurance  carriers.    Members  may  continue  with 
the  plan  on  a  fee-for-service  basis  if  they  leave  their  place  of  employment 
or  purchase  a  nongroup  policy.    Membership  reimbursement  for  3,000  low-income 
group  members  in  the  Mission  Hill  area  is  supplied  by  two  different  contracts, 
one  with  the  Massachusetts  Department  of  Public  Health  and  one  with  the  U.S. 
Public  Health  Service  for  community  residents  whose  income  places  them  just 
above  Medicaid  eligibility.    The  funding  sources  which  support  HCHP  are  dis- 
tributed in  the  following  way: 

90.0%   -  prepaid  capitation 

3.3%    -  fee-for-service  incurred  from  new  members 

2.7%    -  pharmacy  fee 

2.1%    -  dental  fee 

0.01%  -  grant 


MEDICAL  RECORDS  SYSTEM  -  C0STAR 

When  an  individual  selects  HCHP,  the  employer  forwards  copies  of  the 
completed  application  card  to  the  designated  insurance  carrier,  who  fowards 
them  to  HCHP.    The  data  are  received  and  entered  into  C0STAR  (the  computer) 
via  the  CRT  by  a  clerk  in  the  enrollment  department.    A  sequentially  ordered 
medical  record  unit  number  is  generated  by  the  computer  for  each  new  enrol  lee. 
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Each  time  a  face-to-face  encounter  between  patient  and  provider  takes 
place,  an  encounter  form  is  completed.    This  encounter  form  is  the  data  source 
for  any  and  all  compilation  of  patient  information. 

For  a  routine  visit,  the  encounter  form  identifying  data  (e.g.,  patient 
name,  unit  no.,  date,  site,  type  of  visit  and  provider  name)  are  filled  in  by 
clerical  personnel  before  the  provider  sees  the  patient. 

This  information  (key  punched  by  clerical  personnel)  is  transmitted  through 
the  keyboard  of  the  CRT  directly  connected  to  the  computer  which  also  monitors 
the  accuracy  of  the  data.    This  use  of  check  letters  on  codes  reduces  the  chance 
of  transcription  error. 

The  provider  completes  the  vital  signs  and  the  checklist  relating  to  demo- 
graphic information  and  has  the  option  of  including  up  to  180  characters  of 
free  text  about  a  patient's  personal  and  social  background  at  the  bottom  of  the 
encounter  form.    When  extensive  notation  is  required,  the  provider  may  dictate. 
This  dictation  may  be  retrieved  in  problem-linked  fashion  either  in  flow  chart 
format  or  as  part  of  the  encounter  form.    The  amount  of  dictation  varies  from 
department  to  department.    For  example,  only  3%  of  the  pediatric  encounters  entail 
dictation  while  the  departments  of  Neurology  and  Orthopedics  use  dictation  ex- 
tensively.   In  Internal  Medicine  approximately  one-third  of  the  encounters  had 
dictation  averaging  a  length  of  12  lines. 

The  status  of  all  diagnoses  and  medications  can  be  further  specified  by 
the  type  of  indicator  mark  used  (e.g.,  diagnoses  are  characterized  as  major, 
minor,  presumptive,  rule  out,  status  post  or  inactive).    Space  is  provided  to 
enter  diagnoses  and  medications  that  do  not  appear  as  precoded  options.  These 
are  coded  by  record  room  personnel  using  a  full -code  directory  (a  structured 
list  of  problems,  medications  and  tests  which  have  been  encountered  in  the  past 
seven  years  of  the  Plan's  existence). 

For  automatic  monitoring  of  the  patient's  care,  the  provider  checks  the 
"follow-up  important"  box  on  the  encounter  form.    If  the  visit  does  not  occur, 
appropriate  reports  are  generated. 

Information  is  stored  on  random  access  disk  tapes.  The  medical  records 
room  supplies  a  computer  print-out  of  any  or  all  parts  of  the  medical  record. 
Data  display  can  take  the  form  of  an  encounter  form,  flow  chart  or  status 
report.  The  status  report  is  the  basic  data  display  format.  It  is  produced 
for  all  scheduled  visits  and  is  the  most  frequently  used  display  option.  It 
is  designed  to  provide  an  index  to  and  summary  of  the  current  medical  status 
of  the  patient. 

According  to  the  COSTAR  booklet,  the  use  of  check  letters  on  codes  not 
only  reduces  transcription  error  (less  than  1%)  but  has  resulted  in  better 
communication  between  providers  and  record  room  personnel  and  fewer  problems 
with  illegible  handwriting.    There  is  a  technical  problem,  however,  in  that 
only  the  portion  of  the  record  that  is  precoded  can  be  used  for  analysis. 
Therefore,  there  is  no  analysis  of  information  contained  in  the  free  text 
or  dictation. 
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COMPUTER  FACILITIES 


COSTAR  is  a  highly  sophisticated  computer  system  which  allows  a  great  deal 
of  information  to  be  pulled  and  charted  by  a  small  number  of  personnel.  It 
is  an  information  and  communication  system  designed  to  meet  the  needs  of  a  pre- 
paid group  practice  with  regard  to  the  membership  for  new  enrollments  as  well  as 
terminations.    The  computer  is  capable  of  generating  multiple  copies  of  provider 
schedules  and  selected  information  regarding  patients  who  are  to  be  seen.  Data 
may  be  retrieved  by  patient  name,  unit  number  or  family  number;  the  unit  number 
is  most  commonly  used.    If  only  the  name  is  known,  it  may  be  typed  into  the  com- 
puter and  a  search  of  the  data  base  will  display  all  names  matching  or  coming 
close  to  the  entry. 

The  computer  also  provides  all  data  required  for  the  daily  operation  of 
plan  management,  budgeting  and  planning  for  future  growth.    To  facilitate  the 
billing  process,  COSTAR  Supports  computer  to  computer  communication  (via  mag- 
netic tape)  which  coordinates  membership  information  on  file  at  the  Plan  and 
its  insurance  carriers.    In  addition,  it  generates  bills  for  fee-for-service 
patients  and  Medicare  bills  for  patients  over  65  years  of  age. 

It  has  taken  about  15  programmer  years  to  develop  COSTAR  to  its  present 
status.    The  computer  facility  at  LCS,  occupying  8,000  square  feet,  operates 
five  similar  machines  (one  is  available  for  back-up).    COSTAR  is  a  large  col- 
lection of  computer  programs  written  in  MUMPS.    The  system  is  supported  by  a 
PDP-15  computer  system  (48K  18-bit  words  of  core,  260  million  characters  of 
on-line  disk  storage  supporting  30  simultaneous  users).    In  addition  to  the 
system,  there  are  more  than  100  printers  and  cathode-ray  terminals  at  LCS. 

Remote  printing  and  video  terminals  are  used  at  HCHP-Kenmore.    Two  KSR-33 
teletypes  are  located  in  the  medical  record  room.    Two  ITT  video  terminals  and 
14  Infoton  video  terminals  are  also  located  in  the  record  room,  laboratory, 
Departments  of  Internal  Medicine,  Medical  Specialty,  Pediatrics,  Surgery, 
0B-GYN,  Triage,  administration  and  the  main  desk. 

A  provider  Acceptability  Study  conducted  at  the  Kenmore  Center  indicated 
90%  of  physicians  preferred  the  computerized  system  to  a  manual  one. 


PHYSICAL  LAYOUT 

The  Kenmore  Center  occupies  four  and  one-half  floors  (approximately  45,000 
sq.  ft.)  of  a  12-story  apartment  building  in  the  Kenmore  Square  section  of 
Boston. 

HCHP  has  13  waiting  areas,  approximately  55  examination  rooms,  approximately 
35  physician  offices,  a  diagnostic  X-ray  department  with  two  equipped  rooms,  a 
limited  diagnostic  laboratory,  two  minor  operating  rooms,  a  pharmacy,  a  six-chair 
dental  suite,  a  computer  area,  a  test  data  center  and  a  parking  garage.  In 
addition,  a  supervised  playroom  is  available  for  children. 
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AFFILIATIONS 


HCHP  has  several  affiliations  with  other  institutions: 

1.  HCHP  staff  has  a  contract  with  the  Laboratory  of  Computer  Science 
for  the  management  of  the  computerized  medical  records. 

2.  HCHP  staff  members  have  joint  appointments  at  the  Harvard  Medical 
School . 

3.  HCHP  staff  conducts  various  research  projects  with  staff  at  the 
Harvard  Center  for  Community  Health  and  Medical  Care. 

4.  Staff  members  have  various  responsibilities  for  teaching  and  pro- 
viding medical  care  at  Boston  Hospital  for  Women,  Beth  Israel 
Hospital,  Cambridge  Hospital,  Children's  Hospital  Medical  Center 
and  Peter  Bent  Brigham  Hospital. 

The  HCHP  quality  assurance  program  represents  a  high  level  of  variable 
activity,  with  more  activity  actually  going  on  than  was  documented.  Although 
it  is  difficult  to  describe  a  systematic  administrative  or  methodological  frame- 
work for  the  activities,  the  major  clinical  departments  (through  representatives) 
seemed  very  active  in  quality  assurance  projects  and  studies.    The  QAC  opera- 
tional role  seems  to  be  one  of  central  coordination. 
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SECTION  IV 

H  C  MS     STUDY  INFORMATION 


HCHP  agreed  to  participate  in  the  study  via  a  telephone  conversation  on 
November  3,  1975.    Initial  documentation  pertaining  to  quality  assurance  acti- 
vities was  received  on  November  6,  1975.    A  detailed  set  of  Quality  Assurance 
Committee  minutes  was  received  on  November  28,  1975. 

HCMS  staff  visited  the  Harvard  Community  Health  Plan  (HCHP)  on  February  3, 
1976.    The  documentation  received  and  compiled  prior  to  the  site  visit  is  as 
fol lows: 

1.  Part  II-Completed  Data  Collection  Instrument 

2.  HCHP  Handbook 

3.  HCHP  Organizational  Chart 

4.  Membership  Report  (October  20,  1975) 

5.  Provider  Encounter  Report  by  Specialty  for  each  Center  (October,  1975) 

6.  Handbook  describing  the  development  and  structure  of  COSTAR 

7.  Research  grant  application  to  Public  Health  Services  for  the  Implemen- 
tation of  a  Computer-Based  Ambulatory  Quality  Assurance  Program 

8.  Entire  set  of  Quality  Assurance  Committee  meeting  minutes 

9.  Operating  statement  (summary  page)  for  the  fiscal  year  ending  Septem- 
ber 9,  1975 

10.    HCHP  booklet  providing  extensive  data  for  the  new  planned  facilities 

The  site  visit  was  conducted  on  February  3,  1976.    Site  investigators 
could  arrange  visits  with  only  two  HCHP  personnel.    The  interviews  were  conduc- 
ted jointly,  lasting  approximately  five  hours.    Those  interviewed  were  the 
Medical  Director  of  HCHP  and  the  HCHP  Physician  and  Principal  Investigator 
of  the  HCHP  quality  assurance  grant  submitted  to  the  Public  Health  Service. 

The  Medical  Director  is  responsible  for  monitoring  the  quality  of  medical 
services  in  the  Plan.    He  partly  fulfills  this  responsibility  by  insuring  that 
quality  assurance  activities  are  carried  out  according  to  sound  research  methods 
and  in  accordance  with  the  goals  and  desires  of  the  entire  medical  staff.  The 
HCHP  Physician  is  primarily  responsible  for  implementing  the  research  activities 
in  quality  assurance.    He  will  assume  chairmanship  of  the  QAC  in  FY  1977. 

The  topics  discussed  in  the  interview  were  the  following: 

1.  The  development  of  quality  assurance  activities 

2.  External  and  internal  constraints  in  the  development 

3.  Functions  of  the  Quality  Assurance  Committee 

4.  Quality  assurance  activities  in  which  the  greatest  amounts  of  time  and 
energy  are  expended 

5.  COSTAR  decisions 

6.  Organization  and  staff  functions  at  HCHP 

7.  Specific  quality  assurance  activities 
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8.  Advantages  and  disadvantages  in  certain  quality  assurance  methods 

9.  Philosophy  of  the  participants  with  regard  to  the  QAP 

10.    Management  issues  in  implementing  quality  assurance  activities 
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GROUP  HEALTH  ASSOCIATION,. INC. 
District  of  Columbia 


SECTION  I 
INTRODUCTION 


Group  Health  Association,  Inc.  (GHA)  is  a  prepaid  mul tispecial ty  health 
plan  which  provides  services  to  approximately  100,000  enrollees.    Located  in 
the  District  of  Columbia,  the  plan  offers  services  in  four  ambulatory  facili- 
ties through  approximately  100  physician  staff  members  and  contracts  with 
Doctors  Hospital  for  provision  of  inpatient  services. 

In  operation  since  the  1930's,  GHA  provides  services  to  a  large  group 
of  subscribers,  including  a  sizeable  contingent  of  government  employees.  An 
elected  Board  of  Trustees  establishes  policy  and  oversees  GHA  operations. 
As  a  result  of  recent  changes,  the  medical  director  reports  to  the  executive 
director  who  in  turn  reports  directly  to  the  Board.    Several  committees  estab- 
lished by  the  Trustees  review  enrol  lee  reports  and  set  policy  for  administrative 
personnel.    The  executive  director  is  responsible  for  all  administrative  concerns, 
while  the  medical  director  is  responsible  for  all  medical  staff  concerns,  referral 
cases  and  ambulatory  activities. 

The  Medical  Council  formally  established  a  Quality  of  Care  Committee  (QCC) 
in  1973.    The  medical  director  and  the  QCC  chairman  play  key  roles  in  the  acti- 
vities of  the  committee,  and  interact  closely  with  the  person  who  supplies 
staff  support  to  the  committee's  activities. 

GHA  was  included  in  the  study  because  of  the  operational  status  of  some 
of  their  quality  assurance  activities. 

The  GHA  formal  quality  assurance  program  (QAP)  consists  of  audits  using 
both  implicit  and  explicit  criteria.    Another  part  of  the  program  requires 
formal  terms  of  employment  for  physicians  regarding  educational  standards, 
quality  of  care,  and  a  probationary  period  of  employment. 
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SECTION  II 

QUALITY     ASSURANCE  PROGRAM 


This  section  describes  GHA's  quality  assurance  activities  in  terms  of 
developmental  background  and  documented  plans,  and  operational  program  ele- 
ments.   Information  presented  here  is  based  on  GHA  documents  outlining  specific 
quality  of  care  issues  and  the  approach  to  administration  of  a  quality  assurance 
program. 


BACKGROUND  AND  FORMALLY  PLANNED  ACTIVITIES 

Since  its  establishment  in  the  1 930 ' s ,  GHA's  primary  goal  has  been  to 
deliver  high  quality  care  at  reasonable  cost.    To  do  so,  three  basic  procedures 
have  been  established: 

1.  Consumer  input  through  the  Board  of  Trustees,  which  has  been  estab- 
lished by:    (a)  Election  of  Board  members  by  GHA  members,  (b)  Member 
Advisory  Committee,  (c)  Active  resolutions  from  members. 

2.  A  formal  screening  process  for  hiring  physicians.    First,  there  are 
standards  specifying  educational  requirements  for  physicians  in 
each  department  at  GHA.    Second,  the  prospective  medical  staff  mem- 
ber is  i nterviewed  by  the  Medical  Director,  the  Chairman  of  the 
Department  the  physician  is  applying  to  work  in,  and  by  other  staff 
physicians.    Finally,  the  new  physician's  work  and  relationships 
with  staff  are  reviewed  during  the  first  18  months  by  a  senior  depart- 
ment member  and  the  Medical  Director.    If  the  new  physician  does  not 
meet  designated  standards,  employment  is  terminated. 

3.  The  group  practice  form  of  organization  helps  establish  high  quality 
care  because  medical  records  are  shared,  resulting  in  an  informal 
peer  review.    Because  specialists  are  available  in  the  same  facili- 
ties as  the  general ists,  there  is  more  opportunity  for  sharing  knowledge 
through  consultation.    Since  GHA  does  not  operate  fee-for-service,  there 
is  no  danger  of  "losing"  clientele  by  referral  to  specialists. 

The  Quality  of  Care  Committee  (QCC)  was  established,  according  to  documents, 
to  provide  a  formal  administrative  focus  for  quality  assurance  activities.  The 
committee  is  designed  to  discuss  "significant  issues  raised  in  the  reviews" 
and  to  decide  if  and  when  some  type  of  feedback  action  for  a  specific  provider 
is  necessary. 

After  the  QCC  was  established  in  1973,  a  formal  audit  review  process  was 
developed : 

1 .    To  provide  physicians  with  educational  feedback  from  peers  on  the 
qual ity  of  care;  and 
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2.    To  identify  and  correct  problems  which  interfere  with  effective 
delivery  of  high  quality  health  care. 

Documents  note  that  each  full-time  member  of  the  medical  staff  is  involved 
in  the  peer  review  process.  Each  receives  one  medical  chart  to  audit  every  two 
weeks.    QCC  meetings  are  open  to  all  GHA  physicians. 


AUDIT  REVIEW  PROCESS 

Documents  provided  a  description  of  specific  activities  within  the  audit 
review  process.    This  type  of  QAP  focuses  on  the  audit  of  medical  services  pro- 
cesses.   The  specific  steps  are  listed  below. 

1.  Cases  for  review  are  randomly  selected  by  such  indicators  as  deaths, 
regular  appointments  in  various  departments,  referrals  to  other 
specialties,  etc. 

2.  The  goal  is  to  review  40  charts  every  two  weeks  using  physician  re- 
viewers.   Reviewing  physicians'  reports  are  anonymous.    Only  the  pro- 
fessional assistant  who  pulls  the  case  knows  to  whom  it  is  sent. 

3.  The  review  is  typed  and  sent  with  the  chart  to  the  primary  physician. 
If  there  is  no  primary  physician,  the  review  is  sent  to  the  physician 
who  last  saw  the  patient. 

4.  The  primary  physician  comments  on  the  same  sheet  as  the  reviewer, 
responding  to  reviewer  comments,  and  clarifying  elements  of  care  un- 
recorded in  the  chart  or  misinterpreted  by  the  reviewer. 

5.  Completed  reviews  are  sent  to  a  QCC  member  in  batches  of  15.  The 
member  summarizes  trends,  makes  suggestions  and  reports  findings 
to  the  committee. 

6.  The  QCC,  comprised  of  both  physicians  and  nonphysicians ,  discusses 
significant  issues  raised  in  the  reviews.    They  suggest  constructive 
action  when  necessary. 

7.  Minutes  are  taken  at  the  meeting,  typed,  and  approved  by  the 
chairman  of  the  committee. 

8.  The  minutes  of  each  meeting  are  sent  to  all  full-time  physicians 
and  to  regular  part-time  physicians  in  all  specialties. 

Documents  used  by  study  staff  to  ascertain  planned  quality  assurance 
activities  also  reflected  some  results  of  QCC  activities.    The  particular 
method(s)  employed  to  gain  reported  results  were  not  described;  however,  some 
results  were  described  as  follows: 

1.  Asked  for  and  received  from  the  Department  of  Radiology  reports 
dealing  with  radiation  safety  procedures  for  members  obtaining 
X-rays;  this  was  also  done  with  the  dental  test 

2.  Investigated  the  merits  of  IVPs  in  hypertension  that  can  be  con- 
trolled by  drugs 

3.  Investigated  problems  relating  to  precautionary  measures  for  immu- 
nization of  pregnant  women 
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4.  Secured  legal  advice  on  how  long  during  night  hours  patients  in 
the  emergency  visit  center  may  be  kept,  in  the  District,  as  well  as 
in  Maryland  and  Virginia 

5.  Reviewed  proper  usage  of  Vitamin  B12 

6.  Invited  other  physicians  to  discuss  specific  problems 

7.  Coordinated  QCC  with  the  Laboratory  Committee,  Medical  Records  Com- 
mittee and  administrative  personnel  who  relate  to  problems  of  quality 
of  patient  care,  e.g.,  Medical  Records  Department 

The  documents  further  describe  recurring  quality  assurance  issues  identi- 
fied by  6HA  personnel,  including: 

1.  Inadequate  annotation  of  records  on  face  sheets,  particularly  important 
diagnoses 

2.  Lack  of  hospital  data  in  the  medical  record 

3.  Need  to  reeducate  patients  who  continually  use  the  Minor  Injury  Unit 
(MIU)  instead  of  primary  physicians 

4.  Need  for  improved  legibility  of  notes 

5.  Need  for  carefully  documented  notes  for  legal  purposes,  as  well  as 
quality  of  care 

6.  Need  for  follow-up  when  patient  makes  no  attempt  to  continue  care, 
particularly  true  of  the  parent  in  Pediatrics 

7.  Need  for  prompt  reporting  of  important  abnormal  findings  from  the 
X-ray  Department  and  the  Laboratory  Department  to  the  physician 

8.  Need  for  complete  history  sheet  on  all  family  members 

9.  Need  for  growth  and  development  charts  in  all  pediatric  records 

10.  Need  to  assure  records  are  edited  properly  for  appropriate  patient 
care 

11.  Lack  of  communication  between  departments 

According  to  documentation,  GHA's  philosophy  of  health  care  delivery  is 
based  "on  the  concepts  of  quality  care,  maintained  standards  for  the  qualifi- 
cations of  personnel,  and  established  mechanisms  so  that  both  providers  and 
consumers  have  input  to  the  identification  and  resolution  of  problems."  GHA 
hopes  to  assure  quality  of  care  more  effectively  by  "implementing  a  formalized 
system  of  linkages  between  assessment,  standard  setting,  problem  resolution  and 
evaluation  of  effects  on  care  and  outcome." 


OPERATIONAL  ACTIVITIES 

Although  the  documents  outlined  a  general  quality  assurance  program,  the 
present  (1975)  operational  components  of  the  program  were  ascertained  during 
site  interviews  and  review  of  documents  reflecting  its  operations. 

This  description  of  operational  activities  came  from  several  primary 
sources:    (a)  1975  Annual  Report  (quality  assurance  program),  (b)  1976  Quar- 
terly Report  (January  to  March,  1976),  and  (c)  three  sets  of  QCC  minutes. 
Study  staff  received  minutes  from  three  Quality  of  Care  Committee  meetings 
held  in  February  and  March  of  1976,  as  well  as  the  QCC  1976  Quarterly  Report. 

The  chairman  of  the  QCC  plays  a  major  role  in  decision  making.  He  takes 
a  strong  interest  in  deciding  what  to  review,  the  type  of  review,  and 
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subsequent  action  based  on  review.    He  interacts  with  medical  staff  and  directs 
administrative  personnel  in  determining  what  issues  will  be  examined  in  the  con- 
text of  the  QAP.    All  quality  assurance  decisions  emanate  from  the  medical  staff, 
although  the  administrative  side  has  input  in  terms  of  resource  commitment  and 
support  of  QAP  activities.    The  executive  director  has  no  real  interface  with 
the  program. 

Significant  support  comes  from  the  Medical  Records  Department  which  is  in 
charge  of  data  gathering  procedures  for  the  QAP.    According  to  a  member  of  the 
Medical  Records  Department,  funding  for  the  program  is  internalized  in  the  medical 
records  budget.    The  executive  director  reported,  however,  that  there  is  no  line- 
item  funding  for  the  QAP,  although  costs  are  carried  in  the  Medical  Records 
Department,  which  budgets  the  cost  of  pulling  charts  and  the  professional  assistant 
for  quality  of  care.    There  is  also  support  from  physician  members  of  the  medical 
staff  who  conduct  medical  review  assessments,  especially  in  the  departments  of 
Pediatrics,  Obstetrics  and  Gynecology  and  Internal  Medicine.    Physician  members 
of  the  QCC  are  invited  to  serve  on  the  committee;  meetings  are  held  during  the 
lunch  hour  to  minimize  physician  time  away  from  patient  care. 

The  QCC  accepts  input  from  various  sources.  One  staff  member  reported  that 
if  a  physician  (usually  department  chief)  requests  an  item  be  examined,  there  is 
usually  100%  follow-through  by  the  committee. 

The  quality  assessment  method  focuses  on  examination  of  a  particular  patient- 
physician  encounter,  although  QCC  does  not  try  to  define  practice  by  individual 
physicians,  but  focuses  on  general  problems  affecting  the  quality  of  care  at  GHA. 


Specific  Audit  Activities 

Items  for  review  are  selected  by  the  medical  director  or  the  QCC  after 
particular  indicators  for  examining  charts  are  selected,  or  when  there  is  a 
problem  which  the  medical  director  or  the  committee  wants  to  examine.  After 
a  problem  has  been  determined  and  indicators  chosen,  a  sample  of  charts  is  pulled 
based  on  those  indicators. 

There  are  no  systematic  sampling  procedures.    The  QCC  Chairman  sets  a  sam- 
ple size  according  to  what  he  thinks  would  yield  reliable  data.    The  medical 
record  is  the  data  source  for  all  quality  assurance  audits. 

The  professional  assistant  for  quality  of  care  examines  approximately  40 
charts  for  any  particular  audit.    The  charts  are  pulled  by  medical  record  room 
personnel  and  given  to  the  professional  assistant.    She  audits  each  chart  for 
10  elements  prior  to  studying  the  same  chart  for  provider  review: 

1.  Medical  History  sheet  in  chart? 

2.  Problem/Diagnosis  sheet  in  chart? 

3.  Diagnoses  entered  on  Problem/Diagnosis  sheet? 

4.  Is  there  an  Anthropometric  Chart  (for  pediatric  charts)? 

5.  Is  there  a  primary  physician? 

6.  Used  MIU  almost  exclusively  during  the  past  year? 
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7.  Is  patient  being  seen  regularly  in  nonprimary  care  specialty? 

8.  Date  of  enrollment  with  GHA? 

9.  Has  blood  pressure  been  recorded  in  the  past  3  years? 

10.    Has  a  pap  test  been  done  in  the  past  2  years  (for  female  patients 
over  age  25)? 

The  professional  assistant  has  full  responsibility  for  data  collection 
procedures  and  handles  procedural  details  that  include  the  following: 

1.  Notes  what  sample  size  must  be  pulled 

2.  Conducts  a  clerical  audit  of  charts 

3.  Forwards  medical  records  for  physician  review  and  insures  that  the 
physician  conducts  the  review 

4.  Compiles  results  from  both  reviews 

She  has  certain  criteria  from  which  to  select  based  on  the  indicator  or  problem 

chosen.    For  review,  the  criteria  for  evaluation  of  the  records  are  developed 

by  the  QCC  (sometimes  input  from  the  clinic  is  used  to  develop  specific  criteria) 
to  be  used  in  two  audit  areas: 

1.  For  the  review  of  charts  by  both  professional  and  nonprofessional 
personnel 

2.  For  a  specific  problem  or  indicator  chosen 

After  charts  are  checked  for  the  ten  elements  listed  above,  they  are  sent 
to  a  physician  who  reviews  the  chart  and  comments  generally,  using  his  profes- 
sional judgment  to  note  both  excellent  and  deficient  care.    The  eight  questions 
the  reviewing  physician  must  answer  are: 

1.  Were  all  health  problems  suggested  by  data  in  the  record  properly  iden- 
tified? 

2.  Were  the  medical  data  in  the  chart  (history,  physical  findings,  lab 
tests,  etc.)  adequate  to  support  logical  clinical  decisionmaking? 

3.  Was  the  logic  of  care  sound,  i.e.,  did  the  clinical  process  "make 
sense"  to  the  auditor  and  were  the  diagnostic  and  therapeutic  actions 
reasonable,  necessary,  timely  and  thorough? 

4.  Was  follow-through  adequate  regarding  all  important  health  problems? 

5.  Was  prompt  action  given  to  all  important  problems? 

6.  Did  the  patient  benefit  from  the  care  (i.e.,  were  satisfactory  outcomes 
achieved)?    If  not,  why? 

7.  Additional  comments 

8.  If  there  is  no  primary  physician,  does  the  record  suggest  current  health 
care  needs  which  are  not  being  taken  care  of?    (If  "yes"  is  checked,  staff 
will  arrange  assignment  to  and  follow-up  by  a  primary  physician.) 

These  eight  questions  are  followed  by  "General  Comments  on  Record  Itself": 

1.  Is  chart  in  order  and  complete? 

2.  Legible? 
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After  the  physician  completes  the  review,  the  medical  record  is  forwarded 
to  the  professional  assistant.    The  assistant  then  types  the  review  comments, 
attaching  them  to  the  medical  record,  and  sends  both  to  the  primary  physician 
for  clarification  of  elements  of  care  that  may  not  have  been  recorded  and  figured 
in  the  reviewing  physician's  assessment.    The  primary  physician  is  also  required 
to  note  whether  any  patients  must  be  notified  of  problems  revealed  during  the 
review.    The  physician  returns  the  record  to  the  professional  assistant. 

Prior  to  each  QCC  meeting,  members  are  presented  with  15  to  20  reviews  to 
study.    They  analyze  the  reviewing  comments  and  report  their  findings  to  the 
committee  as  a  whole  at  the  regular  biweekly  meeting.    The  committee  then  dis- 
cusses the  reviews  focusing  on  issues  to  be  addressed.    From  review  findings, 
the  committee  may  decide  to  take  action  based  on  the  committee's  consensus. 
The  responsibility  to  take  action  based  on  the  evaluation  rests  with  the  depart- 
ment head,  the  medical  director,  and  the  Quality  of  Care  Committee.    Minutes  of 
the  meetings  are  recorded,  typed,  and  distributed  to  all  regular  GHA  staff 
physicians. 

According  to  interview  information,  if  a  problem  with  a  particular  physi- 
cian is  discovered,  some  administrative  action  based  on  peer  audits  reviewed 
by  the  Quality  of  Care  Committee  is  usually  necessary  to  inform  that  physician 
he  has  not  performed  satisfactorily.    The  committee  discusses  the  problem,  then 
takes  it  to  the  department  head  for  feedback  to  the  physician.    The  committee 
does  not  have  a  direct  line  of  communication  or  a  direct  responsibility  for 
feedback  to  the  audited  physician.    The  only  systematic  format  for  feedback  is 
sending  the  review  to  the  primary  physician  before  it  is  sent  to  the  committee. 

The  1975  Annual  Report  documented  18  QCC  meetings  between  January  and  Decem- 
ber, 1975.    Reviews  of  391  medical  records  were  conducted  using  the  following 
indicators: 

1 .  Hospital ization 

2.  Adult  medical  visits 

3.  Pediatric  visits 

4.  OB-GYN  visits 

5.  Deaths 

6.  Special  study 

7.  Request  of  staff  physician 

This  report  did  not  specify  any  results  of  QCC  actions  or  physician  reviews. 

The  Quarterly  Report  (January  to  March,  1976)  indicated  105  reviews  were 
conducted;  major  QCC  comments  related  to  illegibility  of  handwriting,  lack  of 
a  primary  physician,  and  technical  chart  deficiencies  (lack  of  a  problem  sheet, 
hospital  discharge  summary,  etc.).    Committee  members  present  results  of  reviews 
done  by  themselves  or  a  non-QCC  staff  member,  and  their  assessments  of  each  case. 
In  general,  comments  did  not  reveal  serious  deficiencies  in  care.    The  minutes 
also  indicate  that  when  problems  were  identified  in  primary  physician  service, 
the  physician  was  contacted  regarding  review  results.    The  QCC  member  reported 
that  either  the  primary  physician  had  taken  some  action  or  action  had  been  recom- 
mended through  another  party,  usually  the  department  head. 
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QCC  members  presented  approximately  15  cases  per  meeting  and  some  type  of 
feedback  activity  was  mentioned  for  most  cases. 


Special  Studies 

The  QCC  not  only  reviewed  charts,  but  also  analyzed  data  from  three  special 
studies: 

1.  There  was  a  follow-up  on  150  blood  sugar  tests  selected  randomly  from 
the  lab  files.    Results  were: 

a.  One  hundred  patients  had  elevated  blood  sugar.    Ninety-seven  were 
known  to  be  diabetic;  ninety-four  were  receiving  therapy;  one  was 
a  new  diagnosis;  one  had  another  diagnosis.    Only  one  was  not  a 
diabetic  and  no  explanation  for  the  elevated  blood  sugar  was  re- 
corded . 

b.  In  10  hospital  patients,  only  one  was  a  known  diabetic.  Follow- 
up  showed  elevated  blood  sugar  in  the  other  nine  was  due  to  use 
of  intravenous  fluids. 

2.  "A  review  was  conducted  of  the  yield  [in  terms  of  significant  comments 
by  reviewer  and/or  primary  physician]  by  various  indicators  used  to 
select  206  charts  for  review.    In  general,  from  22%  to  37%  of  charts 
reviewed  contained  significant  comments  with  pediatric  and  'other' 
indicators  having  the  highest  yield.    In  the  'other'  category,  as  might 
be  expected,  57%  of  those  selected  for  review  because  of  death  were 
found  to  have  significant  comments.    Currently,  the  committee  is  in- 
vestigating the  use  of  specific  diagnosis  (e.g.,  alcoholism)  as  an 
indicator  to  assess  the  usefulness  of  this  approach  rather  than  ran- 
dom selection." 

3.  The  third  study  dealt  with  blood  pressure  readings.    Results:  77% 
had  had  blood  pressure  recorded  in  the  past  two  years.    Based  on  this 
finding,  the  medical  director,  with  committee  approval,  made  the  fol- 
lowing recommendations: 

a.  It  is  the  nursing  staff's  responsibility  to  take  blood  pressure 
readings  routinely  for  all  patients  who  come  in  for  appointments 
if  they  have  not  had  one  in  the  past  year. 

b.  Regular  announcements  should  be  published  in  the  GHA  News  en- 
couraging members  to  have  blood  pressure  measured  annually,  even 
if  they  do  not  have  occasion  to  attend  GHA  for  medical  services. 

Other  quality  of  care  meetings  have  focused  on  issues  dealing  with  the 
development  of  new  objectives  and  directions  for  the  committee,  as  follows: 

1.  The  committee  discussed  assessing  the  quality  of  care  of  GHA  members 
at  a  particular  hospital. 

2.  The  committee  would  like  to  see  review  procedures  occurring  on  indi- 
vidual departmental  levels  to  assess  the  quality  of  care  in  each 
department.    If  that  were  to  occur,  the  QCC  would  then  act  as  a 
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coordinator  among  the  various  departments.    Primary  responsibility 
of  the  QCC  would  be  special  studies  and  quality  of  care  issues  that 
concern  GHA  as  a  whole. 

3.    The  committee  discussed  other  procedures  for  quality  assurance,  such 
as  the  tracer  method  and  the  use  of  diagnoses  as  indicators  for  chart 
reviews. 


Staff  for  Quality  Assurance 

The  Quality  of  Care  Committee  is  comprised  of  the  medical  director,  the 
chairman  (selected  by  the  Medical  Council),  representatives  from  various  depart-  I 
ments  (these  have  included  Internal  Medicine,  Dermatology,  Psychiatry,  Urology, 
Pediatrics,  OB-GYN),  a  registered  nurse,  the  director  of  the  Medical  Records 
Department,  and  one  of  the  assistants  to  the  medical  director.    Physicians  from 
outlying  centers  can  participate  in  meetings  by  telephone  network.    This  committee1 
is  purely  a  reviewing  committee  that  makes  suggestions  and  recommendations  to  the 
medical  staff  for  improving  the  quality  of  care. 

Besides  the  professional  assistant  for  Quality  of  Care,  20  people  in  the 
medical  records  room  play  a  part  in  the  QAP.    They  are  responsible  for  filing 
and  retrieval,  and  have  no  special  training  for  QAP;  the  number  of  records  they 
pull  for  review  is  a  very  small  part  of  their  total  workload. 


ADDITIONAL  COMMENTS 

To  assure  higher  quality  of  care,  GHA  has  several  plans: 

1.  To  implement  a  Medical  Care  Information  System  (MCIS)  based  on  encounter 
forms  with  detailed  information  regarding  the  patient's  problem  and 
care.    When  this  system  is  in  complete  operation,  it  will  make  the  fol- 
lowing activities  possible: 

a.  Selecting  charts  for  the  audit  of  care  by  specific  diagnosis 

b.  Producing  medical  problem  lists  for  the  medical  record  and  for 
patients  seen  in  locations  where  the  medical  record  is  available 

c.  Notifying  patients  of  recalled  pharmaceutical  products 

d.  Identifying  potential  adverse  drug  interactions 

2.  To  use  qualified  specialists  outside  GHA  to  audit  the  small  departments 
such  as  Ophthalmology  and  Surgery. 

3.  To  examine  and  restructure  the  present  audit  process  in  order  to  estab- 
lish a  more  formal  link  between  assessing  the  quality  of  care  and  the 
problem  solving  activities.    This  link  would  lead  to  the  assurance  of 
quality,  and  would  accomplish  three  things: 

a.  The  departments  would  have  more  responsibility  for  clinical  issues 

b.  Interdepartmental  issues  would  be  handled  by  a  central  committee 

c.  Nonphysician  clerical  personnel  would  be  used  more  efficiently  in 
the  auditing  process 
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SECTION  III 


DELIVERY     SYSTEM  CHARACTERISTICS 


GHA,  Inc.,  is  a  prepaid  mul tispecial ty  practice  serving  an  enrol  lee  popu- 
lation of  more  than  100,000  with  a  provider  staff  of  approximately  100  physicians. 

GHA  contracts  with  Doctors  Hospital  for  most  inpatient  services.  Provisions 
in  the  plan  allow  for  hospitalization  of  GHA  patients  requiring  tertiary  care  at 
other  inpatient  facilities  in  the  metropolitan  area,  including  university-affiliated 
hospitals  and  the  major  pediatric  hospital  in  the  District  of  Columbia. 

The  Board  of  Trustees  is  responsible  for  all  GHA  operations.    The  Board 
operates  through  numerous  committees  (e.g.,  Claims,  Executive,  Membership,  etc.) 
which  receive  information  from  enrol  lees.    The  committees  report  to  the  trustees. 
The  executive  and  medical  directors  also  report  to  the  trustees,  as  well  as  inter- 
acting with  committees. 

GHA  provides  preventive,  diagnostic  and  therapeutic  care,  primarily  in  GHA 
centers  and  in  hospitals;  home  calls  are  made  when  necessary.    Services  include: 

1.  Diagnosis  and  treatment  of  illness  or  injury 

2.  Checkups,  well -baby  care,  and  immunizations 

3.  Surgery,  special ist  care,  physical  therapy 

4.  Eye  care  (includes  examinations  for  eye  glasses  and  examinations  for 
prescription,  but  not  fitting,  contact  lenses) 

5.  Diagnostic  and  lab  tests,  X-ray  tests,  and  treatments 

6.  Visiting  nurse,  ambulance  and  outpatient  hospital  services  when  au- 
thorized by  a  GHA  doctor 

7.  Immunizations  for:    rubella,  polio,  measles,  smallpox,  cholera,  typhus, 
influenza,  cold,  mumps,  diphtheria,  whooping  cough,  tetanus  and  typhoid 

8.  Surgery  for  congenital  defects  of  children  born  to  GHA  members 

Maternity,  mental  illness,  out-of-service-area ,  and  hospital  benefits  are 
also  offered. 

The  GHA  medical  record  is  based  on  a  traditional  chronologically-ordered 
system.    The  record  includes  registration  information,  clinical  progress  notes, 
laboratory  data,  X-ray  and  medication  information.    Each  GHA  member  has  a  medical 
record,  kept  in  the  facility  where  he  chooses  to  seek  his  primary  care.    An  addi- 
tional feature  contributing  to  the  quality  of  care  is  the  quality  of  the  record, 
which  is  promoted  by  a  central  dictation  system  physicians  use  to  complete  pro- 
gress notes  in  the  medical  record. 
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SECTION  IV 

HCMS     STUDY  INFORMATION 


Staff  contacted  GHA  through  correspondence  dated  October  20,  1975.  GHA 
agreed  to  participate  in  a  telephone  discussion  on  November  14,  1975.  Initial 
information,  received  December  5,  1975,  is  listed  below: 

1.  General  description  of  the  Quality  of  Care  Program  and  the  Quality 
of  Care  Committee  (March  25,  1974) 

2.  Flow  chart  of  quality  of  care  process  at  GHA  (no  date) 

3.  Presentation  made  to  Montgomery  County  Health  Planning  Advisory  Board 
on  March  17,  1975,  entitled  "Quality  of  Care  Assurance  at  GHA" 

4.  The  Quality  of  Care  Review  Summary  which  is  the  Basic  Audit  Abstracting 
Sheet  (no  date) 

5.  Report  on  Blood  Pressure  Study  (1976) 

6.  Quality  of  Care  Quarterly  Report  (1/3/76) 

7.  1975  Annual  Report 

8.  Minutes  of  Quality  of  Care  Committee  meetings  (1976) 

9.  Bylaws  of  GHA,  Inc.  (January  1,  1964) 

10.  GHA,  Inc.,  Board  Regulations  (Pursuant  to  GHA  Bylaws  Article  VI,  Sec- 
tion E)  (January  1  ,  1970) 

11.  GHA,  Inc.,  Schedule  of  Services,  payment  plans  and  subscription  charges 
for  individual  members  (plan  revisions  effective  1/1/75) 

12.  General  description  of  GHA  Comprehensive  Group-Practice  Plan,  1976 

13.  Newsletter  "GHA  News"  Annual  Report  Issue,  April-May,  1975,  Vol.  XXXVI, 
No.  2 

The  documentation  provided  a  general  description  of  the  QAP  and  future  goals; 
however,  there  was  little  documentation  of  specific  quality  assurance  methods 
within  the  present  program  operations.    Information  regarding  these  activities 
was  gained  from  two  sources  primarily:    (a)  onsite  interviews,  (b)  review  of 
Quality  of  Care  Committee  (QCC)  minutes.    Site  visits  were  conducted  on  January  29 
and  April  13,  1976,  at  which  time  interviews  were  conducted  with  the  following 
personnel : 

1.  Medical  Director 

2.  Professional  Assistant  for  Qual ity  Assurance 

3.  Director  of  Medical  Records 

4.  Member  of  Quality  of  Care  Committee  and  OB-GYN  Department 

5.  Executive  Director,  GHA 

Interviews  with  the  first  three  were  audio  recorded. 

The  medical  director  plays  a  key  role  in  the  direction  and  activity  level 
of  the  Quality  of  Care  Committee  (QCC).    The  professional  assistant  is  responsible 
for  all  data  collection  and  compilation  for  quality  assurance.    She  works  with 
medical  record  room  personnel,  and  often  consults  with  the  supervisor  of  the 
department. 


122 


KAISER  PERMANENTE  MEDICAL  CARE  PROGRAM  OF  NORTHERN  CALIFORNIA 

Oakland,  California 


SECTION  I 

INTRODUCTION 

The  Kaiser  Permanente  Medical  Care  Program  of  Northern  California  is  a 
large  prepaid  medical  delivery  system  offering  inpatient  and  outpatient  services 
to  an  enrolled  subscriber  population.    The  medical  care  program  is  divided  into 
four  basic  organizations:    (a)  the  Kaiser  Foundation  Health  Plan,  (b)  Kaiser 
Foundation  Hospitals,  (c)  the  Permanente  Medical  Group,  and  (d)  Permanente 
Services,  Inc.    The  key  contractual  relationships  are  those  of  the  Kaiser 
Health  Plan  with  the  Permanente  Medical  Group  and  Kaiser  Foundation  Hospitals. 
The  Northern  California  region  includes  12  hospitals,  each  with  an  affiliated 
ambulatory  clinic.    There  are  also  four  detached  ambulatory  clinics,  each  with 
an  organizational  tie  to  a  hospital. 

The  Kaiser  program  was  included  in  the  study  because  of  experience  with 
developing  and  implementing  quality  assurance  methods.    A  very  specific  approach 
and  methodology  have  been  used  in  all  Kaiser  facilities.    This  description 
focuses  on  general  program  development  rather  than  any  one  facility's  activities. 

The  quality  assurance  program,  titled  Comprehensive  Quality  Assurance 
System  (CQAS),  was  first  applied  in  1969  in  one  24-man  department  at  one 
facility  and  went  into  operation  September,  1973,  in  all  facilities.  Within 
each  clinic  the  program  has  undergone  various  changes  and  levels  of  activity 
in  the  subsequent  two- and- a- half  years. 
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SECTION  II 


QUALITY     ASSURANCE  PROGRAM 


The  CQAS  program  developed  from  two  main  sources: 

1.  The  search  for  a  method  to  identify  true  educational  needs 
recognized  by  both  the  Professional  Staff  and  Administration 

2.  The  interest  of  the  Permanente  Medical  Group  (PMG),  in  establishing 
a  sound  methodological  approach  to  a  formal  quality  assurance 
program  which  addresses  problem  areas  and  medical  issues,  not 

just  utilization  or  peer  review 

This  description  will  concentrate  on  drawing  distinctions  between  the 
program's  elements  as  outlined  in  documents,  and  the  operational  status  of  the 
program.    The  CQAS  program  is  designed  to  be  a  systematic  approach  to  quality 
assurance  which  should  be  followed  in  a  specific  implementation  format.  An 
important  aspect  of  the  Kaiser  program  is  the  relationship  between  the  office 
of  the  coordinator  of  education  and  the  CQAS  committees  of  the  various  facilities. 
As  described  by  Kaiser  personnel,  the  cornerstone  of  the  CQAS  program  is  the 
flexibility  retained  by  each  facility  in  implementing  their  programs.  The 
Permanente  Medical  Group  (the  physician  partnership)  officially  endorses  the 
program  and  lends  substantial  support  to  it  in  all  facilities.    Kaiser  Foun- 
dation Hospitals  officially  endorses  and  funds  the  program.    Total  organi- 
zational support  is  evident  from  the  fact  that  no  outside  funding  has  been 
employed  to  support  the  program.    All  costs  are  included  as  regular  operating 
budget  items.    The  importance  of  the  coordinator  of  education's  personal 
commitment  and  influence  cannot  be  underestimated,  as  the  developer  of  the 
approach  and  the  person  who  initially  placed  the  CQAS  program  in  all  12 
facilities.    The  generation  of  interest  and  support  of  the  program  among  phy- 
sicians-in-chief of  facilities  and  leaders  in  the  PMG  was  emphasized  as  a 
critical  step. 

The  most  pertinent  facts  regarding  the  12  facilities  are: 

1.  Each  medical  center  delivers  both  inpatient  and  outpatient  services, 
under  one  facility  administration 

2.  Clinical  departments  within  the  facilities  each  have  a  physician 
department  head  who  reports  to  the  physician-in-chief  of  the  facility 

3.  Each  administrator  reports  directly  to  the  regional  administrator 
of  Kaiser  Foundation  Hospitals  who  in  turn  reports  both  to  the 
regional  manager  and  to  the  executive  director  of  PMG 

4.  Each  administrator  interacts  with  the  physician-in-chief  of  the 
same  center  in  administering  daily  operations 

5.  Each  physician-in-chief  reports  directly  to  the  executive  director 
of  the  Permanente  Medical  Group 

The  office  of  the  coordinator  of  education  has  responsibility  for  Kaiser's 
quality  assurance  program.    Programs  within  each  facility  were  initiated  and 
are  monitored  from  his  office. 
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FORMALLY  PLANNED  ACTIVITY 


This  section  describes  the  formal  outline  of  the  CQAS  program.  The 
following  section  (Operational  Activities)  will  describe  certain  operational 
items,  though  generally  the  data  indicate  that  the  CQAS  program  is  being  im- 
plemented as  planned.    The  program  can  be  divided  into  several  key  areas 

1.  Definition  of  goals  and  objectives 

2.  Organization  of  the  program 

3.  Functions  of  CQAS  committee 

4.  Selection  of  medical  records  for  problem  identification  purposes 

5.  Identification  of  problems  by  audit  and  other  mechanisms 

6.  Establishment  of  audit  standards  to  measure  problem 

7.  Design  of  and  performance  of  clerical  data  collecting  procedure 

8.  Determination  of  corrective  actions  based  on  review 

9.  Report  to  the  Board  of  Kaiser  Foundation  Hospitals  and  Executive 
Committee  of  Permanente  Medical  Group 

The  goals  and  objectives  of  the  program  are  well-defined;  emphasis  is  on 
accomplishment  of  these  goals  by  each  CQAS  committee.    The  importance  of  CQAS 
is  to  show  improvement  measured  by  the  number  of  problems  identified,  standards 
established,  solutions  proposed,  and  actions  taken.    The  documented  goals  and 
objectives  include: 

1.  Assurance  of  high  quality  health  care.    This  involves: 

a.  The  identification  of  those  elements  of  care  being  performed 
suboptimal ly 

b.  Improved  performance  of  those  elements 

c.  Measurement  and  documentation  of  changes  in  the  performance  of 
these  elements  of  care  after  improvement  activities  are 
instituted 

These  issues  are  similar  to  administrative  control  procedures  which 
measure  results  of  system  modifications  and  improvement  activities. 

2.  Cost  containment.  It  is  assumed  by  the  CQAS  program  that  better 
quality  care  is  in  the  long  run  less  expensive  care.    Decrease  in 
the  cost  of  care  with  no  decrease  in  quality  is  a  worthwhile  pursuit, 
not  only  economically,  but  also  because  unnecessarily  expensive  care 
diverts  resources  from  more  pressing  health  care  needs  and  can 
interfere  with  proper  utilization  of  available  resources. 

3.  Satisfaction  of  legislative  and  accreditation  requirements. 
Since  it  is  the  intent  of  a  variety  of  government  and  voluntary 
agencies  to  validate  quality  of  care  and  quality  assurance 
procedures  in  health  delivery  systems,  documentation  necessary 
for  responsible  internal  administration  of  quality  assessment 
should,  if  systematically  performed,  satisfy  external  quality 
assurance  accreditation.    CQAS  documents  note  that  quality 
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assurance  programs  that  aim  to  achieve  external  accreditation 
are  doomed  to  fail  in  improving  quality  because  tokenism, 
avoidance  behavior,  and  "busy  work"  will  thwart  their  intention. 

These  three  goals  are  also  integrated  into  the  impetus  for  CQAS  program 
development  -  the  establishment  of  educational  needs  for  providers,  measurement 
of  success  of  educational  activities  and  quantitative  measures  of  certain 
elements  of  medical  care.    Because  CQAS  originated  within  the  coordinator 
of  education's  office,  it  is  based  on  the  educational  and  improvement  value  of 
certain  quality  measurement  procedures. 

CQAS  goals  and  objectives  were  determined  during  the  development  of  the 
CQAS  program  elements  and  the  initial  implementation  of  the  CQAS  methodology 
in  1969  within  one  department.    During  this  time,  the  procedure  was  primarily 
that  of  problem  identification  in  an  anonymous,  nonpunitive  manner  with  proper 
publication  of  problems  to  staff.    A  revision  took  place  after  this  first 
implementation  of  CQAS  process  was  analyzed;  documentation  indicates  there 
were  two  major  flaws: 

1.  Nonprofessional  help  was  not  utilized,  and  physicians  were 
spending  time  in  routine  tasks 

2.  Objective  measures  of  progress  and  improvement  were  not  available, 
thus  true  progress  could  not  be  monitored 

Kaiser  documentation  indicated  that,  in  1972,  the  program  was  reported  to  be 
"running  smoothly."    To  assess  the  review  process,  "physicians  from  other  medical 
centers  evaluated  the  procedure  by  reviewing  and  comparing  medical  record  entries 
for  a  sample  of  patients."    These  records  were  chosen  from  a  sample  of  patients 
seen  in  fall  1969  and  1971  (the  beginning  of  the  program  and  two  years  later). 
A  specific  visit  in  each  year  was  reviewed  and  compared.    The  reviewing  physicians 
were  asked  to  evaluate  the  two  visits  and  approximate  data    as  "better,"  "same" 
or  "worse,"  using  unanimity  as  a  condition  for  the  first  and  last.    It  was  clear 
that  improvement  occurred  in  many  aspects  of  care,  but  this  comparative  review 
affected  the  CQAS  review  process  as  follows: 

1.  Reviewing  physicians  had  to  make  value  judgments  on  care  for 
noncomparable  elements 

2.  Review  process  could  not  yield  quantitative  evaluation 

3.  Review  process  was  overly  dependent  on  physician  time 

4.  Problems  that  arose  could  not  be  addressed  in  a  systematic 
fashion 

Kaiser  documentation  notes  the  most  significant  conclusion  of  these  events 
was  the  necessity  for  developing  criteria  or  standards  for  elements  of  care 
to  be  reviewed.    In  September,  1973,  the  present  elements  of  the  CQAS  program 
were  implemented.    These  elements  were: 

1.  Problem  identification 

2.  Setting  standards  for  problems  identified 

3.  Screening  by  clerical  personnel  and  validation  of  variations  by 
professionals 

4.  Recommendations  for  corrective  action  and  implementation  of  action 
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5.  Remeasurement  of  compliance  with  same  or  modified  standard 

6.  Reporting  of  results  to  top  management 

7.  Documentation  of  each  step  to  insure  feedback  loop 

The  goals  and  objectives  of  Kaiser  CQAS  were  not  revised,  but  the  process 
for  measuring  and  improving  care  was  changed.    The  documentation  devotes 
over  19  pages  to  conceptual  background  and  philosophy  behind  specific  CQAS 
program  elements. 


Organization  of  Program  and  CQAS  Committees 

The  key  person  in  implementing  the  CQAS  methodology  in  each  facility  is 
the  CQAS  chairperson  (a  physician).    The  documentation  notes  that  the  chairperson 
should  work  closely  with  the  administrative  coordinator  of  the  CQAS  committee. 

The  coordinator  (or  at  times  the  chairperson)  is  responsible  for  several  key  areas: 

1 .  Budget  of  the  CQAS  program 

2.  Coordination  of  various  data  collection  efforts 

3.  Clerical  personnel 

4.  Meetings  for  standard  setting  and  problem  solving 

Each  medical  center  has  a  CQAS  Committee  which  includes  the  chief  executive 
physician  and  chief  lay  administrator,  the  chief  of  education,  the  nursing 
director,  the  medical  records  librarian  and  physician  representatives  from  major 
clinic  departments.    This  committee's  goal  is  to  provide  overall  (clinic-  and 
hospital  -wide)direction  for  the  review  activities  in  several  areas: 

1.  To  receive  information  on  interdisciplinary  problems  identified 
for  review  and  standards  established  by  departments  for  measurement 

2.  To  refer  problems  to  appropriate  department  for  action 

3.  To  determine  the  priority  of  measurement  for  standards 

4.  To  set  standards  which  deal  with  an  entire  facility 

5.  To  affirm  or  modify  data  collection  proposals,  as  well  as  adding 
modifications  or  support  for  specific  corrective  activities 

6.  To  keep  medical  center  personnel  informed 

In  addition  to  the  CQAS  committee,  there  are  separate  clinical  department 
peer  review  committees.    The  importance  of  departmental  committees  centers  on 
the  problem  identification  process  which  is  the  critical  starting  point  for 
the  entire  CQAS  methodology.    Each  departmental  committee,  according  to  documents, 
should  conduct  several  activities: 

1.  Identify  problems  and  receive  information  from  external  sources 
about  problems 

2.  Establish  standards 

3.  Transmit  to  medical  center  CQAS  Committee  information  on  clinical 
review  of  variations  found  in  the  CQAS  clerical  data  collection  results, 
including  specific  problems  found  in  own  and  other  departments 

4.  Recommend  corrective  activities  for  own  department 
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The  CQAS  coordinator  works  with  all  committees,  but  is  responsible  to  the 
medical  center  CQAS  Committee.    The  coordinator  supervises  the  clerical  person 
who  collects  data  for  standards  and  compiles  results.    He  must  also  interact 
with  medical  record  room  personnel  who  pull  charts  for  CQAS  use. 


Medical  Records  Selection  for  Problem  Identification 

1    j  s 

The  documents  identify  "microsampling"  as  the  major  method  for  problem 
identification  although  problems  from  committees  and  complaints  from  any  other 
sources  are  similarly  directed  to  the  CQAS  Committee.    In  microsampling,  medical 
records  are  examined  to  reveal  problems  and  are  usually  chosen  by  selecting  a 
sample  of  patients  who  were  seen  at  certain  hours  of  the  day  and  on  certain  days 
of  the  week,  or  from  other  activity  logs.    The  records  are  selected  after  a 
four-  to  six-week  time  lapse  to  allow  all  laboratory  work  and  follow  up  visits  to  be  1 
recorded.    This  problem  identification  and  selection  method  insures  that  issues 
addressing  most  types  of  care  and  most  providers  will  be  revealed. 

Microsampling  means  that  a  relatively  small  number  of  medical  records  are 
reviewed  for  problem  identification;  documents  note  20  records  per  department 
every  two  months  reveals  a  greater  number  of  issues  than  can  possibly  be  corrected  j 
with  resources  available. 

A  physician  (or  other  appropriate  professional)  performs  problem  identification 
review  using  various  formats.    Documentation  notes  "experienced  reviewers  will 
prefer  the  free  format  of  a  blank  sheet  of  paper,"  although  other  formats  can 
be  used.    Documentation  outlines  several  general  elements  which  should  be 
considered  during  the  review  including: 

1.  An  index  visit  is  reviewed  in  its  entirety  (index  means  the  specific 
visit  that  resulted  in  the  selection  of  the  record) 

2.  The  reviewer  searches  for  "what  is  wrong,  not  who  is  wrong" 

3.  Reviewer  should  note  "anything  in  the  record  which  reveals  or  will 
lead  to  poor  care" 

4.  After  index  visit  is  reviewed,  other  visits  are  scanned  to 
determine  if  there  are  internal  inconsistencies  in  care  of  the 
patient 

5.  Poor  care  and  unnecessarily  expensive  care 

Once  this  initial  review  is  completed,  two  physicians  exchange  the  record 
(with  the  first  reviewer's  comments).    A  second  review  is  conducted  to  catch 
undetected  errors  and  to  corroborate  the  first  reviewer's  comments.  These 
findings  are  presented  to  the  departmental  CQAS  Committee,  which  must  unanimously 
agree  on  the  findings.    Findings  are  then  presented  to  departmental  staff  for 
comment,  discussion,  and  criteria  or  standard  setting  when  appropriate.  At 
this  point  in  the  process,  the  identified  problems  may  be  addressed  by  other 
procedures  besides  standard  setting  and  measurement.    If  the  problem  can  be 
solved  through  means  such  as  education,  system  changes,  etc.,  it  may  be  done 
by  the  department  whether  the  problem  is  compatible  with  setting  a  standard  or 
not. 
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Standard  Setting  for  Performance  Measurement 

While  the  problem  identification  phase  is  designed  to  discover  problems 
(what  is  wrong),  the  standard  setting  phase  is  designed  to  state,  relative  to  a 
specific  problem,  what  should  be  done  (what  is  right).    Documents  suggest 
standards    are  needed  for  two  purposes: 

1.  To  save  professional  time  by  using  clerical  personnel  to  measure 
performance  once  a  problem  has  been  detected 

2.  To  evaluate  corrective  measures 

Once  the  department  or  the  CQAS  Medical  Center  Committee  decides  a  problem 
is  suitable  for  correction  through  the  standard-setting  mode,  the  departmental 
committees  can  then  begin  to  develop  a  standard  for  a  specific  problem  with  help 
as  needed  from  the  Medical  Center  CQAS  Committee.    The  documentation  discusses 
several  different  methods  for  establishing  standards.    A  major  issue  is  that  the 
responsible  department  must  create  its  own  standards.    The  department  members 
must  agree  that  the  standards  reflect  what  might  be  done  in  a  specific  medical 
care  situation.    A  significant  amount  of  implicit  professional  judgment  is 
required  for  developing  each  standard,  which  is  an  important  aspect  of  the 
CQAS  method.    Further,  to  cultivate  commitment  to  CQAS,  standards  must  be 
developed  by  the  group  of  providers  who  are  the  subject  of  measurement. 

The  documentation  does  not  note  any  systematic  format  to  follow  in  standard 
development,  other  than  that  standards  reflect  "what  is  right"  for  a  particular 
care  situation.    Documentation  delineates  several  characteristics  standards 
should  have: 

1.  Should  be  easily  measurable,  consisting  of  a  simple  statement 
relative  to  a  specific  problem 

2.  Must  be  easily  understandable  by  the  clerical  personnel  so  data 
collection  will  be  uniform  and  reliable 

3.  Must  state  a  desired  performance  under  a  specific  set  of  circumstances 

4.  Should  relate  to  a  consequential,  solvable,  and  relatively  prevalent 
problem 

Standard  development  is  designed  to  allow  maximum  flexibility  for  the 
departments  and  medical  centers  as  they  develop  measurable  statements  dealing 
with  medical  care.    These  standards,  which  reflect  a  brief  statement  about 
professional  behavior,  are  submitted  to  the  medical  center  CQAS  Committee 
which  ranks  them  for  measurement.    Departments  do  not  have  separate  employees 
to  perform  data  collection;  it  is  done  throughout  the  medical  center. 


Data  Collection  for  Measurement  and  Recording  of  Performance 

According  to  documentation,  once  standard  development  is  accomplished, 
"the  first  two  phases  of  professional  activity"  are  complete,  and  activity 
centers  on  clerical  data  collection  and  measurement.    Administrative  functions 
are  recorded  on  worksheets,  which  state  the  standard  and  specific  instructions 
for  measuring  performance  related  to  the  standards.    Worksheet  information  also 
includes  the  number  of  variations  of  performance  from  the  standard,  professional 
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validation  of  variations,  recommendations  (if  made)  for  corrective  action  and, 
most  important,  the  evidence  that  recommendations  have  been  implemented.  Work- 
sheets are  handled  by  the  CQAS  coordinator  and  document  the  status  of  audit 
process  in  four  phases: 

1.  Statement  of  standard  and  method  of  data  collection  and  measurement 

2.  Clerical  function  reports,  presenting  results  of  audits  and 
measurement 

3.  Professional  review  and  analysis  of  specific  cases  of  variation 

4.  Analysis,  conclusion,  action  recommended  and  taken 

The  worksheets  are  not  only  used  to  document  activities  through  the  review 
process,  but  also  serve  as  procedural  guides  throughout  the  entire  process. 

Worksheet  I,  which  includes  statement  of  the  standard,  must  have  several 
other  data  elements: 

1.  Medical  center  name 

2.  Standard  number 

3.  The  date  of  CQAS  meeting  when  the  standard  was  first  formulated 

4.  Index  words,  which  identify  the  standard  for  library  purposes 

5.  Source  of  standard  initiation,  usually  a  department 

6.  Name  of  clerical  person  responsible  for  measuring  standard  and 
the  date  referred  to  the  clerical  person  for  measurement 

7.  Whether  any  modification  of  standards  was  done  in  course  of  time, 
from  audit  to  audit 

8.  Interpretation  of  terms 

9.  Authority  for  standard  setting  (usually  a  department) 

10.  Specific  instruction  for  selecting  type  of  data  determined 
necessary  for  measurement  (place,  time) 

11.  Source  of  data  (progress  notes,  nurse's  notes,  diagnosis  con- 
firmation, etc.) 

12.  The  quantity  of  data  (records)  that  will  be  audited  against 
the  standard 

The  clerical  person  responsible  for  conducting  the  measurement  locates 
the  source  and  type  of  data  required  and  reports  to  the  CQAS  coordinator. 

Worksheet  II  must  include  the  following  information: 

1.  Results  of  audit  reported  by  clerical  person  to  CQAS  coordinator 

2.  The  number  of  records  screened,  and  the  number  audited  against 
standard  (these  figures  can  be  different) 

3.  The  number  of  variant  cases,  plus  the  percentage  of  variant 
cases  out  of  those  tested 

4.  The  identification  (by  patient  identification  number)  of 
individual  variant  cases 

5.  Observations,  not  necessarily  requested,  made  by  the  clerical 
person  during  data  collections  which  may  clarify  or  explain 
variations 

6.  General  comments  by  the  clerical  person 
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The  results  presented  within  the  context  of  Worksheet  II  are  then 
distributed  to  a  professional  who  validates  the  variations.    Analysis  is 
presented  to  the  CQAS  committee.    The  responsibility  for  analysis  rests  with  the 
appropriate  department  or  the  medical  center  personnel  who  are  the  subject  of  the 
standard. 

Worksheet  III  documents  the  analysis  of  variations  identified  in  clerical 
reports.    The  analysis  centers  on  whether  there  exists  some  clinical  or  admin- 
istrative justification  for  documented  variation.    Data  included  on  the  worksheet 
during  this  analysis  phase  are: 

1.  Date  when  the  results  of  the  review  and  analysis  were  reported 
to  CQAS  committee 

2.  The  identification  number  of  the  variant  case 

3.  A  brief  description  of  the  type  of  variation 

4.  A  "yes"  or  "no"  as  to  whether  the  variation  was  justified.    A  "no" 
response  indicates  that  corrective  action  is  desirable,  while 

a  "yes"  response  indicates  the  case  will  be  dropped  at  that  point 

5.  Percentage  of  measured  and  reviewed  cases  not  justified 

6.  For  cases  not  justified,  data  must  be  entered  which  notes  where  the 
variation  took  place  and  where  corrective  action  should  be  directed 
(three  categories  are  listed,  "institutional,  department,  individual") 

7.  The  reviewer  must  be  identified,  with  accompanying  comments  on 
type  of  problem  and  resolution  (if  any)  of  the  variant  case 

The  documentation  did  not  specify  any  guidelines  to  be  followed  for  review 
of  justifications.    This  determination  is  left  to  the  implicit  judgment  of  the 
reviewer. 

Worksheet  IV  notes  the  type  of  action  recommended  and  actually  taken.  The 
data  elements  required  here  are: 

1.  The  data  analysis  conclusions  (Worksheet  III)  reported  to  the  CQAS 
committee 

2.  Conclusion  of  the  CQAS  analysis  stated  and  the  type  of  need 
(education,  channeled  system,  etc.)  identified 

3.  Date  that  recommendation  was  sent  to  action  agent 

4.  Brief  statement  describing  the  type  of  action  recommended 

5.  Person  who  has  responsibility  for  acting  on  the  recommendations 
(e.g.,  department  chief) 

6.  Date  that  action  was  taken 

7.  The  type  of  action  that  was  instituted  (time,  place,  results) 

The  CQAS  coordinator  is  responsible  for  all  worksheet  data  and  insures  each 
step  is  correctly  matched  with  the  required  data  elements  on  the  worksheets. 
Although  the  process  from  Worksheets  I  through  IV  is  systematic,  the  CQAS 
system  depends  upon  flexibility  and  judgment  in  deciding  whether  a  variant  case 
is  justified. 

In  summation,  the  discussion  above  describes  the  two  major  points  of  the 
CQAS  program: 

1.    A  review  of  medical  records  (or  other  documentation  or  activities) 
to  identify  problems 

131 


2.    The  method  for  quantitatively  assessing  the  problem  and  documenting 
that  corrective  action  has  been  implemented,  remeasuring  to  assess 
the  success  of  the  corrective  action.    This  is  a  six  phase  process: 

a.  Deciding  whether  a  problem  is  amenable  to  standard  setting 
format  for  solving 

b.  Developing  a  standard  to  measure  a  problem 

c.  Stating  the  standard,  collecting  information  on  the 
standard  and  measuring  the  problem  against  the  standard 

d.  Analysis  of  measurement  (specific  variation  compared  to 
standard)  to  determine  whether  a  justification  exists 

e.  Presenting  conclusion  of  analysis 

f.  Recommending  appropriate  corrective  action  and  following 
through  to  see  that  it  is  successfully  implemented  and 
corrects  variation 


OPERATIONAL  ACTIVITIES 

This  section  of  the  program  description  examines  the  Kaiser  quality 
assurance  program  from  one  perspective:    the  implementation  of  CQAS  through 
the  coordinator  of  education's  role  for  the  Permanente  Medical  Group.  In- 
vestigators asked  to  review  two  individual  CQAS  programs,  but  agreed  to  the 
suggestion  such  review  would  not  allow  for  an  accurate  perspective  of  the 
overall  program. 

The  office  of  the  coordinator  of  education  has  just  added  its  first 
central  staff  member  to  formally  monitor  and  assist  the  individual  CQAS 
programs.    Previously,  staff  consisted  only  of  a  secretary.    As  noted  earlier, 
the  CQAS  program  originated  with  the  coordinator  of  education  but  was  imple- 
mented by  personnel  involved  at  various  facilities.    Decentralized  implemen- 
tation was  deliberate  although  the  coordinator  of  education's  office  did  moni- 
tor and  assist  with  different  program  activities. 

The  operational  status  of  all  the  medical  center  CQAS  programs  was 
determined  through  two  processes: 

1.  Abstraction  of  CQAS  meeting  minutes  for  different  medical 
centers 

2.  Interview  documentation  from  Meg  Kellogg 

Presently,  12  medical  centers  have  implemented  CQAS  activities.  The 
operational  status  of  each  CQAS  program  seems  to  vary  by  facility  because  of 
the  individualized  implementation  and  will  be  described  by  facility,  based 
on  a  review  of  CQAS  committee  meeting  minutes. 

Investigators  examined  all  of  the  minutes  compiled  at  the  office  of  the 
coordinator  of  education  to  note  general  activities  implemented  by  the  dif- 
ferent facilities.    The  minutes  were  examined  only  to  substantiate  program 
operations.    Minutes  from  each  facility  reflected  the  following  general  activities 
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Facility  A 


1.  Two-hour  CQAS  meetings  took  place 

2.  Meeting  covered  action  on  20  standards 

3.  CQAS  committee  to  send  memorandum  to  each  department  peer  review 
meeting  regarding  standard  setting 

4.  Use  of  Worksheets  I,  II,  III  and  IV  was  noted 

5.  Discussion  of  CQAS  budget 

6.  Discussion  of  setting  centerwide  standard  and  the  routing  for 
approval 

7.  Eighty-eight  standards  were  at  different  stages  of  implementation 
Facility  B 

1.  Reformulation  of  several  standards 

2.  Audit  results  of  two  standards  presented 

3.  Discussion  of  standards  developed  by  Pediatric  Department 

4.  CQAS  subcommittee  appointed  to  further  study  and  develop  one 
standard 

5.  Identification  of  one  problem  within  X-ray  department 

6.  Discussion  of  summarizing  and  documenting  departmental  audits 

7.  Development  of  standards  for  improvement  of  progress  notes 

8.  Discussion  of  new  standards  from  Department  of  Medicine 

Facility  C 

1.  Review  of  all  standards  developed  in  facility 

2.  Discussion  of  five  centerwide  standards 

3.  Attempt  to  establish  standards  for  treatment  of  acute  gonorrhea; 
subcommittee  established  to  study  the  problem 

4.  Discussion  of  treatment  for  pelvic  inflammatory  disease 

5.  Discussion  of  Nursing  Quality  Assurance  Project 

6.  Report  of  first  meeting  of  Chart  Audit  Committee  for  nursing  services 
Facility  D 

1.  Reports  on  two  pilot  audits,  (1)  head  trauma  (2)  patient  discharge 

2.  Discussion  of  working  with  area  PSRO 

3.  Results  of  pediatric  asthma  audit 

4.  Results  of  nursing  audits 

5.  Report  of  reaudit  of  standard  for  abnormal  pap  smears 

6.  Report  of  reaudit  of  fetal  heart  tones  in  delivery  room 

Facility  E 

1.  Discussion  of  new  telephone  message  form,  with  input  from  CQAS 
committee 

2.  Results  of  audit  for  two  nursing  standards 

3.  Development  of  one  new  standard 

4.  Report  on  two  surgery  standards,  noting  high  compliance 
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Facility  F 


1.  Report  on  utilization  of  packed  cells 

2.  Report  that  administrative  CQAS  Committee  working  on  plans  for 
recording  blood  pressure  in  certain  departments 

3.  Report  on  medical  record  review  (problem  identification) 

4.  Report  that  Nursing  Service  completed  two  audits 

5.  Consideration  of  proposed  audit  goals  presented  by  one  physician 

Facility  G 

1.  Presentation  and  review  of  standard  for  recording  postoperative 
diagnosis 

2.  Report  on  audits  conducted  on  five  standards 

3.  Report  of  a  standard  audit  in  another  facility 

4.  Discussion  of  follow-up  process  for  urine  cultures 

5.  Discussion  of  interaction  with  regional  laboratory  personnel 
regarding  ordering  of  laboratory  tests 

6.  Review  of  CQAS  budget 

Facility  H 

1.  Review  of  five  specific  standards  for  reformulation 

2.  Discussion  of  results  of  five  standard  measurements 

3.  Three  standards  deferred  for  action 

Facilility  I 

1.  Discussion  of  role  of  CQAS  committee  within  medical  center 

2.  Discussion  of  CQAS  methodology 

3.  Development  of  one  new  standard 

Facility  J 

1.  Discussion  of  progress  in  measuring  four  new  standards 

2.  Development  of  one  new  standard 

Facility  K 

1.  Report  of  results  of  four  audits 

2.  Development  of  11  new  standards  for  audit 

The  study  staff  reviewed  all  minutes  compiled  by  the  regional  office. 
Although  the  minutes  were  variable  in  terms  of  format,  content  and  production, 
they  provided  evidence  that  the  outlined  CQAS  methodology  was  being  implemented. 
Minutes  for  the  medical  center  CQAS  committee  not  compiled  in  this  office 
exist,  and  are  compiled  at  each  facility.    CQAS  committee  members  are  not 
required  to  report  all  activities.    Formal  monitoring  does  not  include  minutes, 
but  only  a  sample  of  audit  worksheets  and  periodic  summary  reports,  which  are 
used  to  computerize  the  standards  and  their  status  and  to  report  to  the  KFH 
Board  of  Directors  (no  fixed  reporting  schedule). 
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A  review  of  the  CQAS  committee  minutes  reflects  several  points  regarding 
the  operational  status  of  CQAS  methodology: 

1.  CQAS  Committee  meetings  are  fairly  well  attended,  although  this 
fluctuates  between  high  participation  facilities  and  low 
participation  facilities  (identified  by  Kaiser  personnel). 

2.  The  parameters  of  CQAS  committees  vary  by  facility;  some  committees 
are  involved  only  in  CQAS  matters,  while  others  involve  themselves 
in  administrative,  general  research  and  utilization  issues. 

3.  The  establishment  of  standards  and  the  audit  of  those  standards 
occur  in  all  facilities. 

4.  There  is  a  high  level  of  interaction  between  Medical  Center  CQAS 
committees  and  departmental  committees  (noted  in  10  sets  of  minutes). 

5.  There  exists  a  high  variability  in  the  number  of  standards  used  in 
each  facility  (one  facility  covering  over  20  standards  in  a  meeting, 
another  facility  covering  two  standards  in  the  same  period  of  time). 

6.  Some  facility  nursing  services  are  involved  in  CQAS  activities. 

7.  CQAS  activities  seem  to  cover,  with  equal  probability,  ambulatory 
and  hospital  problems. 

8.  There  seems  to  be  consistency  in  the  phrasing  of  standard  statements 
among  all  facilities. 

9.  All  minutes  reflect  discussion  involving  audit  results  of  standards 
and  development  of  new  standards. 

10.  Minutes  reflected  little  discussion  of  variance  justification  or 
action  taken  on  nonjustified  variances,  although  Kaiser  personnel 
indicated  the  bulk  of  variance  discussion  is  to  be  done  at  depart- 
mental level,  where  variance  is  justified  and  actions  take  place. 

11.  The  frequency  of  notations  about  variance,  justifications,  actions, 
and  developed  standards  in  CQAS  minutes  is  as  follows: 

a.  Variations  -  15 

b.  Justifications  -  2 

c.  Actions  -  6 

d.  Developed  standards  -  109 
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Kaiser  personnel  made  several  points  regarding  CQAS  administration  and 
methodology: 

1.  Introduction  of  more  formalized  and  routine  monitoring  of  facility 
CQAS  activities  in  order  to  make  reporting  more  routine. 

2.  In  terms  of  CQAS  committee  membership,  it  was  noted  that  a  repre- 
sentative of  each  major  department,  director  of  nursing,  physician 
in  chief,  chief  of  education,  administrator  (or  representative), 
and  the  assistant  administrator  (who  is  CQAS  coordinator)  are  the 
usual  members. 

3.  Medical  record  room  personnel  provide  in- kind  support  to  all  CQAS 
activities  in  pulling  charts;  however,  CQAS  personnel  often  avoid 
having  charts  pulled  by  using  other  data  sources  or  obtaining  the 
charts  from  other  places  (e.g.,  at  nursing  stations,  physician 
offices)  depending  on  the  nature  of  the  problem  being  studied. 

4.  The  clerical  personnel  conduct  the  audits,  compile  the  information 
and  type  the  data. 

5.  The  CQAS  clerk  (50  to  100%  time)  and  coordinator  (10  to  20%  time) 
are  budgeted  for  each  facility. 

6.  Physician  time  and  committee  meeting  time  are  not  budgeted 
separately  in  the  formal  budgeting-accounting  system. 

7.  Some  problems  identified  through  CQAS  are  handled  by  administrative 
personnel . 

8.  The  organizational  parameters  of  the  standard  setting  (e.g.,  medical 
center-wide,  department,  etc.)  vary  by  facility. 

9.  CQAS  standards  can  relate  to  both  hospital  and  clinic  problem  areas. 

10.  More  standards  are  developed  within  departments  than  across 
departments . 

11.  Most  CQAS  committees  meet  once  a  month,  usually  for  1  1/2  to  2  hours 
some  meet  weekly,  some  twice  a  month. 

12.  The  CQAS  coordinator  is  supervised  by  the  administrator  for  routine 
activities,  but  reports  to  chairperson  of  CQAS  committee  for  CQAS 
activities. 

13.  CQAS  chairperson  leadership  is  an  important  variable  in  operational 
level  of  CQAS  committees. 

14.  Worksheets  I  through  IV  are  employed  in  most  facilities;  the  few 
exceptions  do  use  worksheets  in  conducting  audits. 
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15.  Regional  meetings  of  facility  CQAS  representatives  take  place  to 
examine  standards  established  across  facilities,  discuss  regional 
problems,  etc. 

16.  The  effectiveness  of  setting  facilities  standards  has  "not  really" 
been  determined. 

17.  No  formalized  and  structured  guidelines  for  standard  development 
exist,  though  the  standards  are  systematic  for  measurement  of 
variations. 


Additional  Comments 

Activity  level  reflects  high  use  of  the  CQAS  program  within  the  12  medical 
centers,  although  there  is  great  variation  among  programs.    The  design  of  the 
CQAS  methodology  allows  discretion  in  the  implementation  of  the  total  program; 
the  major  components  of  the  method  seem  to  be  uniformly  implemented  in  eight 
of  the  facilities. 

Two  areas  in  which  operational  activities  could  not  be  documented  through 
minutes,  but  could  through  the  audit  worksheets  included: 

1.  What  actions  were  taken  on  variations  identified 

2.  Physician  recommendation  on  whether  or  not  variations  were  justified 

The  differences  among  centers  in  several  areas  were: 

1.  What  problems  are  chosen 

2.  The  number  of  standards  generated 

3.  Identification  of  variations 

4.  Amount  of  action  implemented 
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SECTION  III 

DELIVERY     SYSTEM  CHARACTERISTICS 


The  Kaiser  Permanente  Medical  Care  Program  (KPMCP)  is  directed  by  a  17- 
member  board  of  directors.    There  are  four  major  organizations  within  Kaiser: 
(a)  Kaiser  Foundation  Hospitals  (KFH);  (b)  Permanente  Services,  Inc.  ( PS  I ) ; 
(c)  Kaiser  Foundation  Health  Plan  (KFHP) ,  and  (d)  the  Permanente  Medical 
Groups  (PMG),  the  physician  organizations  which  contract  to  provide  profes- 
sional services.    The  contracts  tie  the  medical  groups  to  the  health  plan, 
consisting  of  benefit  packages  and  charges  that  constitute  the  KPMCP. 

The  Kaiser  Permanente  Medical  Care  Program  provides  inpatient  (hospital 
and  long  term  care)  and  outpatient  services  to  a  prepaid,  enrolled  population. 
The  Health  Plan  is  the  financial  component. 

Specific  organizational  positions  include  the  following:    (a)  executive 
directors  of  the  PMG,  (b)  six  regional  presidents  of  KFH  and  KFHP,  who  report 
directly  to  the  board  of  directors;  (c)  senior  vice-president-Northern 
California  regional  manager  reporting  to  the  president.    This  senior  vice- 
president's  office  handles  contractual  relations  with  PMG  for  provision  of 
services.    The  regional  manager  has  several  organizational  components  reporting 
to  him:    (a)  vice-president  and  regional  administrator,  KFH;  (b)  vice-president 
and  manager,  KFHP;  (c)  regional  controller,  KFH,  KFHP.    The  vice-president  and 
regional  administrator,  KFH,  report  jointly  to  the  regional  manager  and  the 
executive  director,  PMG,  who  is  also  regional  administrator  for  the  clinics, 
run  by  PMG.    The  regional  administrator  has  major  units  reporting  to  him, 
encompassing  the  regional  administration  of  ancillary  services  for  the  medical 
centers.    The  medical  center  administrators  report  directly  to  the  regional 
administrator  and  have  responsibility  also  to  the  physician-in-chief  at  each 
medical  center.    The  center  administrators  are  responsible  for  managing  adminis- 
trative, professional  (in  conjunction  with  the  physician-in-chief)  and  ancillary 
services,  and  all  non-physician  department  heads  report  to  them. 

The  Permanente  Services,  Inc.,  group  is  controlled  by  an  independent  board 

of  directors  representing  KFHP,  KFH  and  PMG.    The  president  of  this  group 

reports  directly  to  the  directors.  As  implied  by  its  name,  it  is  a  service 

organization  dealing  with  planning,  finances,  and  other  service  aspects. 

The  Permanente  Medical  Group  is  managed  by  an  executive  committee, 
appointing  an  executive  director  to  administer  the  group's  activities.  The 
coordinator  of  education  reports  directly  to  the  executive  director.  Each 
facility's  physician-in-chief  reports  directly  to  the  executive  director. 

The  Northern  California  Region  includes  12  medical  centers  and  four 
detached  clinics.    The  total  patient  population  is  approximately  1,252,717. 

Each  facility  uses  a  medical  record  of  its  own  choice.    Study  staff  could 
not  determine  how  many  different  types  of  records  were  used,  but  Kaiser  personnel 
indicated  a  chronological  record  is  the  norm. 
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SECTION  IV 

H  C  M  S     STUDY  INFORMATION 


Kaiser  Permanente  agreed  to  participate  in  the  study  during  a  telephone 
conversation  on  December  5,  1975.    Study  staff  had  previously  received 
initial  documentation  about  the  entire  program.    The  quality  assessment 
program  is  officially  titled  Comprehensive  Quality  Assurance  System  (CQAS). 
The  documentation  received  and  compiled  by  the  study  staff  includes  the 
following: 

1.  "A  Comprehensive  Quality  Assurance  System"  (a  99-page  document 
describing  the  entire  CQAS  program) 

2.  CQAS  Worksheets  (December,  1974) 

a.  Standard  and  measurement  instruction  (I) 

b.  Clerical  function  report  (II) 

c.  Analysis  of  variations  for  cl inical -administrative 
justification  (III) 

d.  Recommendations  and  action  (IV) 

3.  Outpatient  Care  Review  Report  (revised  December,  1973) 

4.  Microsampling  data  sheet 

5.  "Departmental  Comprehensive  Quality  Assurance  Committee,"  flow 
diagram  of  CQAS  process 

6.  Medical  Center  Comprehensive  Quality  Assurance  Committee,"  flow 
diagram  of  CQAS  process 

7.  Organizational  chart,  Kaiser  Northern  California  Region  (July  1, 
1975) 

a.  Kaiser-Permanente  Medical  Care  Program 

b.  Regional  Medical  Centers  Administration 

c.  Permanente  Services,  Inc. 

8.  CQAS  meeting  minutes  for  11  facilities  (19  separate  meeting  results, 
from  January  7,  1975  to  December  17,  1975) 

Documentation  provided  staff  with  an  accurate  account  of  the  planned  CQAS 
program  and  an  indication  of  the  operational  status  of  the  program  within  12 
facilities.    Document  No.  1  (see  list),  outline  for  evaluation  of  the  program, 
specified  program  methods  and  elements,  specific  output  of  the  CQAS  problem 
identification  and  measurement,  and  the  overall  philosophy.    It  also  noted  the 
importance  of  program  implementation  at  the  departmental  level. 
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Kaiser  was  visited  by  study  staff  on  March  18,  1976.    Interviews  were 
conducted  with  the  coordinator  of  education  for  the  Permanente  Medical  Group, 
and  a  staff  associate  responsible,  among  other  duties,  for  covering  the  daily 
operations  of  the  CQAS  program.    These  two  act  in  an  advisory  capacity  to  the 
various  CQAS  committees. 

The  interviews  covered  many  topics  including: 

1.  Initial  development  of  CQAS  methodology 

2.  Initial  administrative  implementation  of  CQAS 

3.  Selection  of  review  entities  in  CQAS 

4.  What  types  of  standards  should  be  employed  in  context  of  CQAS 
program 

5.  How  samples  are  chosen  in  CQAS 

6.  How  to  measure  physician  performance  via  formal  quality  assessment 
methods 

7.  Problems  inherent  in  monitoring  quality  activities  in  different 
settings 

8.  Modification  of  program  elements  during  different  phases  of  operations 

9.  Provider  attitudes  when  formal  quality  activities  are  implemented 

The  interviews  focused  especially  on  the  present  operational  status  of 
the  CQAS  program. 

The  main  area  of  responsibility  of  the  coordinator  of  education  has  been 
development  and  revision  of  methodology,  and  to  some  extent,  monitoring  of 
operation  activities.    The  direction  of  the  staff  associate  is  toward  more 
systematic  monitoring  of  and  assistance  to  the  CQAS  medical  center  activities. 
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HMO- INTERNATIONAL 
Los  Angeles  County,  California 


SECTION  I 
INTRODUCTION 


HMO*- International  (HMOI)  provides  health  care  to  approximately  105,000 
people.    Originally  established  as  a  medical  practice  in  Los  Angeles  in 
1941,  HMOI  had  expanded  into  prepaid  medicine  by  1966.    The  home  office 
provides  financial  and  management  services  to  its  operational  subsidiaries 
and  affiliates  from  central  Los  Angeles.    Subsidiaries  and  affiliates  pro- 
vide medical,  laboratory,  pharmaceutical,  dental,  and  optometric  services 
in  19  facilities  throughout  the  Los  Angeles  area.    A  medical  director 
coordinates  all  medical  field  operations  from  the  home  office  with  the 
assistance  of  an  administrative  team  for  managerial  and  supportive  services. 
Each  of  the  19  facilities  is  supervised  by  a  group  administrator  and  a 
senior  physician. 

The  present  quality  assurance  program  evolved  from  an  audit  of 
immunization  rates  and  the  implementation  in  1972  of  the  outcome-based 
health  accounting  approach  to  evaluation  of  hypertension**.    The  program 
began  officially  July  1,  1975,  with  two  approaches:    (1)  problem-oriented 
peer  review  based  on  explicit  process  and  outcome  criteria,  and  (2)  peer 
review  of  randomly  selected  medical  records  based  on  implicit  and  explicit 
process  criteria. 

The  former  was  designed  to  achieve  and  maintain  high  quality  care  of 
specific  health  problems;  the  latter  was  designed  to  improve  quality  in 
the  management  of  individual  patient  visits  and  to  integrate  individual 
visits  with  total  care  of  the  patient.    Both  types  of  review  provide 
performance  profiles  for  physicians  based  on  objective  data  and  identify 
deficiencies  in  the  care  of  individual  patients  for  corrective  action. 


*    Health  Maintenance  Organization 

**  Williamson,  John  W.:    Outcome  assessment  for  implementing  quality 
assurance  systems.    Quality  Assurance  of  Medical  Care  PHEW  (HSM) 
73-7021,  Feb  1973,  pp  313-328 
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SECTION  II 


QUALITY     ASSURANCE  PROGRAM 


This  section  describes  HMOI  quality  assurance  activities  in  three  areas: 
program  background,  description  of  planned  quality  assurance  activities  and 
description  of  operational  activities. 


PROGRAM  BACKGROUND 

Program  documentation  is  not  extensive.    A  letter  from  the  director  of 
quality  assurance  (December  5,  1975)  outlined  major  program  components, 
while  further  information  about  the  program's  operational  status  was  col- 
lected through  onsite  interviews  and  a  follow-up  interview  with  the  director. 
This  description  simultaneously  outlines  the  program  elements  as  they  were 
planned  and  their  operational  status,  since  all  program  elements  are  in  some 
stage  of  implementation. 

A  quality  assurance  program  was  initiated  at  HMOI  for  two  primary 
reasons: 

1.  State  regulations  require  quality  assurance  programs  in  insti- 
tutions which  serve  a  large  Medicaid  population 

2.  HMOI  desired  to  improve  both  the  quality  and  the  cost  effective- 
ness of  its  health  care  by  instituting  better  quality  controls 
and  by  putting  greater  emphasis  on  prevention  or  early  detection 
of  medical  problems 

One  HMOI  physician  had  a  strong  interest  and  sound  experience  in  quality 
assurance.    Planning  for  the  program  included  agreement  by  HMOI  to  appoint  this 
physician  as  full  time  director  of  quality  assurance  and  to  provide  internal 
funding  to  support  the  program.    The  director  then  developed  some  general  pro- 
gram goals  and  initiated  discussions  with  each  HMOI  facility  regarding  those 
goals.    A  general  literature  survey  was  conducted,  emphasizing  outcome  measure- 
ments.   The  director  discussed  various  methods  with  Dr.  John  Williamson  of 
Johns  Hopkins  University,  who  developed  the  health  accounting  approach.  An 
Educational  Planning  Committee  was  established  consisting  of  a  nutritionist, 
a  health  educator,  the  director  of  Ambulatory  Quality  Assurance,  and  the 
director  of  Hospital  Quality  Assurance  to  oversee  some  of  the  quality  assurance 
activities,  as  well  as  to  implement  special  programs  based  on  results  of  quality 
assurance  studies. 

Quality  assurance  activities  are  managed  by  the  director  and  his  adminis- 
trative assistant,  with  support  from  each  facility's  personnel  when  activities 
are  implemented.    Centralized  management  for  19  independent  facilities  requires 
the  director  to  develop,  implement  and  monitor  the  quality  assurance  activities 
in  coordination  with  each  facility's  personnel. 
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HMOI  personnel  cited  four  primary  goals  for  the  program: 

1.  To  reduce  unnecessary  morbidity  and  control  clinic  disease 
conditions 

2.  To  improve  health  status 

3.  To  insure  preventive  care  as  an  integral  delivery  system 
component 

4.  To  define  patient  needs  for  HMOI 

The  program  also  attempts  to  coordinate  quality  evaluation,  staff  develop- 
ment, patient  education,  and  continuing  education  for  physicians.  The 
goals  and  a  program  philosophy  are  currently  being  documented. 

Quality  of  care  is  defined  in  terms  of  health  status.    To  define 
health  status,  specific  problem-oriented  criteria  are  developed  for  each 
problem  selected  for  review.    The  definition  of  quality  evolves  as  the 
various  criteria  are  met  and  health  status  improves. 

Personnel  involved  in  and  funded  by  the  quality  assurance  program 

are: 

1.  The  director,  who  has  total  program  responsibility 

2.  An  administrative  assistant,  who  abstracts  all  charts  and  con- 
ducts initial  review  of  data  under  supervision  of  the  director 

3.  Two  health  accountants  (partially  supported  through  outside 
funds),  who  are  responsible  for  data  collection  in  the  clinics 
involved  in  the  health  accounting  approach 

4.  The  California  State  Department  of  Health,  Dr.  John  Williamson, 

and  selected  personnel  specializing  in  the  topics  under  investigation, 
all  serving  as  consultants. 

PLANNED  ACTIVITIES 

The  program  will  consist  of  two  major  review  efforts: 

1.  Explicit  audit  -  review  of  abstracted  charts  using  explicit 
process  and  outcome  criteria 

2.  Peer  review  -  review  of  charts  using  implicit  and  explicit 
process  criteria 

The  criteria  to  be  used  in  the  explicit  audits  are  described  below, 
followed  by  a  description  of  the  method  to  be  used  for  peer  review. 


Explicit  Audit  Process 

Six  procedures  will  be  followed  in  the  explicit  audit  process: 
1 .    Problem  Identification 

The  Educational  Planning  Committee  with  input  from  participating 
clinics  will  be  responsible  for  selecting  and  ranking  problems  for 
review.    Problems  are  selected  by  considering  three  major  factors: 
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a.  California  State  Quality  Assurance  Department  regulation 
list  of  22  conditions  which  must  be  reviewed 

b.  Current  literature 

c.  High  potential  for  both  medical  and  cost  impact 

Problem  areas  may  include  medical,  administrative  structure,  con- 
sumer and  educational  concerns.    Once  a  list  of  problems  is  developed  in 
order  of  importance,  it  will  be  sent  to  the  16  participating  clinics  for 
agreement. 

2.  Pilot  Evaluation 

An  audit  protocol  will  be  developed  for  a  problem,  then  applied 
in  one  clinic  to  determine  whether  the  problem  warrants  review  in  all  facilities. 
The  Educational  Planning  Committee  will  review  the  findings  and  if  further 
action  is  warranted,  a  systemwide  program  will  be  implemented. 

3.  Criteria  Development 

Guidelines  for  process  and  outcome  criteria  will  be  discussed  with 
personnel  in  all  clinics.    Specific  criteria  and  standards  will  be  developed 
by  the  quality  assurance  department,  sometimes  in  consultation  with 
experts  in  the  problem  area.    Criteria  will  be  revised  as  necessary  after 
meetings  with  medical  staff  in  the  various  facilities. 

4.  Initial  Assessment  in  Each  Clinic 

Data  will  be  abstracted  from  medical  records  by  using  questions  that 
require  a  yes  or  no  answer  and  involve  no  subjective  judgement.  For 
example,  in  the  hypertension  study,  the  eight  questions  were  as  follows: 

1.  Were  the  blood  pressure  measures  taken  properly  under 
proper  conditions? 

2.  Was  the  diagnosis  of  hypertension  confirmed? 

3.  Were  the  necessary  tests  and  studies  completed  to  identify 
the  presence  or  absence  of  target  organ  damage? 

4.  Were  the  necessary  tests  and  studies  done  to  identify 
underlying,  treatable  causes  of  hypertension  and  co-morbid 
conditions? 

5.  Were  the  necessary  tests  and  studies  done  to  identify  adverse 
reactions  secondary  to  the  medications  used? 

6.  Were  the  necessary  periodic  tests  and  examinations  done  to 
identify  progression  of  the  disease  which  might  warrant 
changes  in  treatment? 

7.  Which  patients  did  not  receive  proper  supervision? 

8.  Which  patients  received  proper  supervision,  but  did  not 
have  controlled  blood  pressures? 

The  health  accountant  will  be  responsible  for  abstracting  data  from 
a  100%  sample  of  medical  records  for  a  designated  period,  and  for  prepara- 
tion of  descriptive  statistics.    The  director  will  review  and  analyze  data 
to  develop  recommendations  for  corrective  action. 
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5.  Feedback 


Results  will  be  presented  to  the  medical  staff  at  each  facility. 
The  senior  physician  of  the  facility  and  the  staff  will  be  responsible 
for  formulating  a  specific  corrective  action  plan.    In  some  cases, 
individual  members  of  the  medical  staff  will  be  contacted  for  individual 
counseling.    Educational  programs  and  other  nonpunitive  appropriate  pro- 
cedures designed  to  correct  deficiencies  should  be  stressed.  Study 
results  will  be  made  available  to  the  HMOI  quality  assurance  committees, 
clinic  medical  directors,  and  the  California  State  Department  of  Health. 

6.  Restudy 

After  a  two-month  period,  deficient  cases  will  be  reassessed  using 
basically  the  same  method  as  the  problem  phase.    If  unsatisfactory  acti- 
vity continues,  meetings  will  be  held  with  medical  staff,  health  educa- 
tors, and  educational  planning  committees  to  determine  further  action. 
This  cycle  will  continue  until  the  problem  is  resolved. 

To  clarify  the  concepts  of  process  and  outcome  criteria,  three 
categories  of  outcome  are  defined: 

1.  Diagnostic  outcome:    audits  to  verify  the  diagnosis 

2.  Therapeutic  outcome:    audits  to  verify  the  use  of  appro- 
priate procedure  according  to  diagnosis 

3.  Functional  outcome:    audits  to  verify  that  health  status 

of  the  patient,  with  respect  to  the  problem,  is  satisfactory 

Completing  this  six-step  audit  process  should  take  approximately 
two  to  five  weeks.    Review  of  audit  results  by  the  director  should  take 
approximately  two  weeks. 

Audit  results  will  be  made  available  for  review  to  quality  assurance 
committees,  medical  directors,  state  auditors,  health  accountants,  and 
the  director  of  quality  assurance. 

The  philosophy  of  HMOI  will  be  to  examine  all  three  types  of  outcome 
(diagnostic,  therapeutic  and  functional)  because  they  are  closely  related. 
There  are  no  plans  at  present  to  conduct  audits  of  functional  outcome. 


Peer  Review 

The  second  HMOI  program  procedure  will  be  peer  review  of  randomly 
selected  charts.    All  physicians  in  each  department  will  review  ten  charts 
(attempting  to  identify  most  recent  three  visits  per  patient)  for  desig- 
nated episodes  of  illness.    Charts  will  be  selected  from  the  patient 
register.    A  record  room  clerk  will  select  randomly  charts  of  patients 
seen  during  the  last  30  days,  often  using  every  tenth  chart  to  get  equal 
distribution  among  morning  and  afternoon  patients.    The  director  will 
analyze  numerical  scores  and  will  pull  charts  randomly  to  determine 
agreement.    The  planned  result  of  peer  review  will  be  the  development  of 
physician  profiles. 
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The  philosophy  behind  the  peer  review  approach  will  be  an  educational 
one.    Physicians  will  not  only  receive  a  copy  of  reviews  of  their  records, 
but  will  become  more  aware  of  standards  of  satisfactory  performance  as 
they  conduct  review.    Physicians  who  cannot  perform  a  review  satisfactor- 
ily may  be  in  need  of  education. 

The  first  pilot  study  of  the  peer  review  system  at  HMOI  has  been 
completed,  and  a  second  pilot  study  will  begin  in  October. 


ACTIVITIES  IN  OPERATION 

The  pilot  study  documented  a  major  explicit  audit  for  hypertension 
conducted  in  ten  HMOI  facilities.    The  study  was  implemented  as  planned 
(see  explicit  audit,  described  above),  although  the  implementation  reflected 
a  higher  degree  of  procedural  specificity  than  did  the  plans. 

Before  implementing  the  total  study,  HMOI  pretested  the  method  in 
one  facility  using  the  health  accounting  system. 

The  three  major  steps  in  the  audit  process  were: 

1.  The  abstracting  of  hypertensive  patient  records  and  explicit 
review  by  the  administrative  assistant 

2.  A  review  of  the  abstracts  by  the  director  for  medical 
judgment 

3.  Presentations  of  audit  results  to  facility  personnel 

Each  facility  generated  a  list  of  patients  with  a  diagnosis  of  hyper- 
tension, usually  from  an  index  card  file  kept  by  the  nursing  stations, 
based  on  patient  visit  logs. 

The  director  indicated  that  a  literature  review  was  conducted  as 
part  of  the  process  for  criteria  development,  and  input  from  HMOI  physi- 
cians and  outside  experts  was  sought.    Review  by  each  facility  resulted 
in  criteria  revision.    Once  a  final  consensus  was  reached,  the  director 
began  sample  selection  and  data  collection. 

After  the  first  level  of  review  was  completed  by  the  administrative 
assistant  using  explicitly  defined  criteria,  initial  criteria  were  sub- 
mitted to  the  facilities  for  rediscussion.    For  the  hypertension  study, 
there  was  no  evidence  criteria  were  changed. 

The  administrative  assistant  abstracted  the  records,  collecting 
seven  data  items: 

1 .  Name  of  patient 

2.  Date  of  last  visit 

3.  Blood  pressure 

4.  Note  that  diagnosis  was  made 

5.  Note  of  treatment  regimen 
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6.  Whether  patient  was  supervised,  satisfactorily  or  unsatisfactorily 
(2nd  level  of  review  by  director) 

7.  Whether  hypertension  was  controlled  or  uncontrolled 

The  objectives  of  this  first  level  of  review  were: 

1.  To  develop  an  initial  deficiency  list  for  a  patient  in  reviewing 
treatment  for  hypertension  based  on  explicit  criteria 

2.  To  collect  and  compile  data 

The  data  from  the  abstracting  were  then  stratified  into  four  categories 
of  status  for  hypertensive  patients: 

1.  Controlled  with  satisfactory  supervision 

2.  Not  controlled  with  satisfactory  supervision 

3.  Control  led,  unsatisfactory  supervision 

4.  UncontrolTed,  unsatisfactory  supervision 

The  director  then  reviewed  the  categorized  data  for  each  clinic, 
using  his  medical  judgment  regarding  data  accuracy  in  each  case. 

A  summary  of  the  hypertension  study  indicated  the  total  number  of 
cases  audited  was  688  in  the  10  facilities,  17  to  131  audits  conducted 
per  facility.    Frequency  rates  of  the  four  data  categories  were  also  given. 

Aggregate  figures  by  facility  and  case-by-case  deficiency  lists 
were  presented  to  chief  physicians  at  each  facility.    Chief  physicians  were 
responsible  for  deciding  what  kind  of  corrective  action  to  take  and  when. 
There  is  some  evidence,  however,  that  corrective  action  was  conducted 
with  input  from  the  director. 

Study  results  were  also  presented  to  the  central  Quality  Assurance 
Committee  and  the  medical  director  of  the  California  Medical  Group,  the 
physician  service  affiliate  in  the  HM0I  organization. 

No  evidence  was  available  dealing  with  specific  feedback  procedures 
implemented  based  on  the  hypertension  study. 

The  required  two-month  reassessment  had  not  been  completed  by  HM0I 
personnel  at  the  time  of  this  report,  although  all  indications  were  that 
a  trial  analysis  noting  the  percentage  of  changes  in  controlled  and 
uncontrolled  population  would  be  completed.    Only  one  disease  condition 
was  chosen  for  audit,  but  documentation  indicates  the  audit  was  conducted 
in  a  rigorous  manner.    The  director  indicated  audit  results  would  be  used 
because  of  the  quality  of  the  results  and  because  of  support  from  personnel 
of  the  facilities  and  HM0I  central  administrative  and  medical  personnel. 


ADDITIONAL  COMMENTS 

HM0I  personnel  outlined  plans  intended  for  implementation  in  the  near 
future,  including: 
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1.  More  extensive  use  of  computerized  data 

2.  Use  of  outside  consultants  to  monitor  reliability  of  the  health 
accountant  and  peer  review  data  collection 

3.  Greater  use  of  health  accountants  for  data  collection  and  analysis 
and  less  use  of  the  quality  assurance  director 

4.  Documentation  of  the  goals,  philosophy  and  quality  assurance 
approach 

5.  Assessment  of  cost-benefit 

6.  Expanded  number  of  audits  using  and  refining  the  present  system 

Several  unique  characteristics  of  HMOI  quality  assurance  activities 
noted  by  investigators: 

1.  HMOI  has  implemented  and  tested  a  quality  assurance  method 
for  a  number  of  facilities,  with  only  two  full-time  people 
(director  and  administrative  assistant)  in  a  period  of  approxi- 
mately six  months 

2.  HMOI  implemented  and  tested  its  system  with  no  explicit  line  item 
funding  other  than  the  salaries  and  related  overhead  of  the 
director  and  administrative  assistant 

3.  Sampling  procedures  required  a  100%  sample  over  a  designated 
period 

4.  The  practice  setting  treats  somewhat  varied  conditions: 

a.  An  unstable  and  mobile  patient  population 

b.  Patients  who  relate  more  to  the  clinics  than  individual 
physicians 

c.  A  fairly  high  degree  of  turnover  in  the  medical  staff 


- 
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SECTION  III 

DELIVERY     SYSTEM  CHARACTERISTICS 

i 

HMO-International,  its  subsidiaries  and  its  affiliated  companies  con- 
stitute a  health  services  organization  which  provides  fee-for-service  and 
prepaid  health  care  through  three  organizations: 

1.  Consolidated  Medical  Systems,  Ltd.  (CMS)  is  a  nonprofit  corporation 
formed  to  contract  with  government  agencies.    In  1975,  CMS  was  the  largest 
prepaid  Medi-Cal  provider  in  California. 

2.  California  Medical  Group  Health  Plan,  Inc.  (CMGHP)  is  a  profes- 
sional medical  corporation  which  does  not  contract  with  the  State  of 
California,  but  with  employers,  unions,  and  other  groups  for  provision  of 
negotiated  health  benefits. 

3.  California  Medical  Group  (CMG)  performs  most  of  the  outpatient 
medical  care  rendered  to  members  of  CMS  and  CMGHP.    CMG  is  a  partnership 
of  physicians  operating  in  19  medical  groups.    In  April  1976,  CMG  employed 
93  full-time  and  81  part-time  physicians  at  19  medical  offices.  Services 
include  the  usual  specialty  areas;  drugs,  family  planning,  eye  care, 
psychiatric  care,  and  prosthetic  appliances.    All  patients  are  assigned  to 
a  family  practitioner  for  initial  evaluation,  then  referred  to  specialists 
as  required.    Routine  appointments  are  available  from  8:00  a.m.  to  6:00  p.m. 
five  days  a  week  and  walk-in  or  emergency  appointments  are  accommodated. 

A  chronological  medical  record  is  filed  in  the  patient's  home  office. 
If  the  patient  uses  another  facility,  a  copy  of  his  medical  record  is 
sent  to  his  home  office. 

The  prepaid  population  served  totals  approximately  105,000,  of  whom 
approximately  43%  are  served  through  commercial  contracts  and  57%  through 
contracts  with  the  State  of  California  under  the  Medi-Cal  Act.    More  than 
50%  of  the  population  is  under  25  years  old  and  consists  of  approximately 
57%  females  and  43%  males. 

Fee-for-service  accounts  for  4%  and  prepaid  contracts  96%  of  funding. 
Revenue  per  year  approximates  $40,000,000. 
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SECTION  IV 


H  CMS       STUDY  INFORMATION 


HMO-International  agreed  to  participate  in  the  study  during  a  telephone 
conversation  on  November  13,  1975,  and  forwarded  initial  documentation  of 
the  quality  assurance  program  on  December  5,  1975. 

Several  documents  were  sent  to  study  staff  for  review: 

1.  System  flow  diagram  of  quality  assurance  program  (December  5, 
1975) 

2.  Letter  containing  historical  development  information  (December  5, 
1975) 

3.  Documentation  on  utilization  patterns,  medical  record  system, 
patient  routing  system,  confidentiality  and  accounting 
(November,  1975) 

4.  HMOI  publications  outlining  corporate  procedures 

HCMS  staff  visited  the  site  on  December  22  and  23,  1975.    The  director 
of  quality  assurance  and  the  HMOI  Coordinator  were  interviewed.  After 
the  on-site  investigation,  there  was  more  data  collection  via  telephone 
interviews  with  the  director. 

During  the  interviews,  which  were  the  main  source  of  information  for 
this  description,  several  areas  were  covered: 

1.  Development  of  approach 

2.  Expressed  objectives  and  goals 

3.  Unique  characteristics  of  HMOI  delivery  system 

4.  Organizational  structure  for  quality  assurance 

5.  Focus  of  audit  review  system 

6.  Data  collection  and  compilation  procedures  in  audit 

7.  Abstracting  procedures  used  in  audit 

8.  Use  of  coding  in  data  compilation 

9.  Data  analysis  used  in  audit 

10.  Use  of  audit  results  in  problem  solving 

11.  Administrative  interactions  between  HMOI  personnel  in  audit 
activities 

12.  Development  and  use  of  medical  criteria 

13.  Specific  review  procedures  in  audit  program 

14.  Future  activities  for  quality  assurance  program 


150 


NORTHEAST  VALLEY  HEALTH  CORPORATION 
San  Fernando,  California 


SECTION  I 
INTRODUCTION 


Northeast  Valley  Health  Corporation  (NVHC)  is  a  nonprofit  prepaid  community 
health  network,  originally  funded  by  0E0,  now  partially  supported  by  DHEW.    It  is 
governed  by  a  24-member  board  of  directors  elected  and  appointed  from  the  sur- 
rounding community.    NVHC  operates  two  facilities  in  Pacoima  and  San  Fernando, 
California.    The  San  Fernando  facility  is  the  participant  in  this  study.  Phy- 
sician, administrative,  and  other  staff  are  employed  by  NVHC. 

NVHC's  quality  review  program,  a  Quality  Assurance  System  for  Ambulatory 
Care  (QASAC),  was  formally  funded  by  the  California  Regional  Medical  Program 
(CRMP)  and  staffed  in  October,  1974.    Although  it  was  funded  specifically  as 
a  demonstration  project,  implementation  of  an  ongoing  quality  assessment  program 
was  also  emphasized.    Explicit  criteria  are  used  in  a  peer  review  system  to  per- 
form an  extensive  process-oriented  and  limited  outcome-oriented  method  of  quality 
assessment. 

NVHC  provides  comprehensive  ambulatory  care  directly  through  its  own  staff 
and  provides  hospital  and  specialty  services  by  contracting  with  institutional 
and  individual  community  providers.    Nearly  all  NVHC  revenue  is  received  on  a 
prepaid  basis  (a  monthly  capitation  rate)  from  the  California  Department  of 
Health,  through  support  by  Title  XIX  (Medi-Cal),  Bureau  of  Community  Health 
Services  (0E0  low  income  enrollees)  and  private  groups  and  individuals. 

The  NVHC  executive  director  has  overall  administrative  responsibility  for 
NVHC  and  is  answerable  to  the  board  of  directors.    The  next  level  of  management 
includes  the  following  positions: 

1.  Deputy  Director  of  Health  Services 

2.  Director  of  Quality  Review  and  Utilization  Control 

3.  Deputy  Director  of  Operations 

4.  Deputy  Director  of  Marketing 

5.  Deputy  Director  of  Planning  and  Development 

The  administrator  for  each  facility  reports  to  the  executive  director  and 
each  deputy  director  has  facility  personnel  reporting  to  him  in  a  particular 
area  of  management  responsibility. 

NVHC  was  included  in  the  study  because  it  had  received  funding  specifically 
for  research  in  quality  assurance. 
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SECTION  II 


QUALITY     ASSURANCE  PROGRAM 


Northeast  Valley  Health  Corporation  (NVHC)  quality  assurance  program  is  called 
"Quality  Assurance  System  for  Ambulatory  Care"  (QASAC).    The  program  is  a  research- 
operations  program  jointly  funded  by  NVHC  and  California  Regional  Medical  Program 
(CRMP) ,  although  initial  funding  came  from  0E0. 

QASAC  is  primarily  a  research  program  geared  to  examine  the  development  of 
medical  review  procedures  and  criteria  by  implementing  these  activities  in  an 
operational  setting.    Some  work  has  been  directed  to  feedback  of  audit  results 
and  evaluation  of  QASAC  impact  by  re-audits  of  selected  conditions.    The  director 
of  quality  review  and  utilization  control  reports  to  the  executive  director  of 
NVHC. 

The  director  of  quality  review  and  utilization  control  was  hired  specifically 
to  implement  the  proposal  made  to  CRMP,  which  included  criteria  development,  audit 
of  records,  tabulation,  analysis  and  presentation  of  audit  results  as  well  as 
development  of  computer  programs. 

Prior  to  October,  1974,  and  receipt  of  the  CRMP  funding,  the  NVHC  quality 
review  activities  consisted  of  a  monthly  audit  of  randomly  selected  charts.  Pro- 
viders were  divided  into  teams,  each  team  reviewing  the  charts  of  another  team. 
Participation  in  this  review  was  limited  to  family  practice  physicians  and  family 
nurse  practitioners.    Chart  audits  were  intended  to  include  analysis  of  the 
following  variables: 

1.  Completeness  of  initial  assessment  of  patient  health  status 

2.  Completeness  of  physical  examination  and  medical  history  entries 

3.  Completeness  of  diagnostic  work-up  for  specified  diseases 

4.  Validation  of  diagnosis  made  by  review  of  pathological  tissue  reports, 
pre-  and  postoperative  diagnosis,  admission  and  discharge  diagnosis 

5.  Validity  and  accuracy  review  of  primary  diagnostic  information  such 
as  laboratory  and  radiological  procedures 

6.  Preventive  management 

7.  Use  of  other  therapeutic  techniques 

8.  Appropriateness  of  treatment  including  prescribed  medications 

9.  Timeliness  of  treatment 

10.    Sufficient  follow-up  care  including  recognition  and  notation  of  abnor- 
mal laboratory  and  physical  findings,  proper  patient  education  using 
verbal  and  written  instruction,  proper  referral  of  patient  to  other 
providers 

Reviewed  charts  were  recalled  in  later  audits  to  observe  if  deficiencies 
had  been  corrected. 

This  program  was  severely  limited  by  the  following  factors: 

1.    Review  activities  had  to  be  limited  to  one  hour  per  month,  resulting 
in  a  low  number  of  charts  actually  reviewed 
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2.  There  was  little  agreement  among  providers  on  review  standards  because 
implicit  standards  were  used  by  each  practitioner 

3.  There  was  an  insufficient  number  of  cases  for  each  diagnosis  to  identify 
patterns  of  deficiencies 

4.  There  was  no  measurable  improvement  in  patient  care 

As  a  general  result  of  the  program's  ineffectiveness,  NVHC  submitted  a  grant 
proposal  to  CRMP  for  development  and  implementation  of  a  formal  quality  assurance 
program.    This  grant  was  awarded  to  NVHC  in  October,  1974.    The  documentation 
notes  several  goals  and  objectives  of  the  grant: 

1.  To  develop,  implement,  evaluate  and  refine  a  quality  assurance  pro- 
gram for  an  acute  disease  state,  a  chronic  disease  state  and  a 
preventive  or  health  maintenance  service  (due  to  50%  budget  cut, 
CRMP  and  NVHC  agreed  to  reduce  quality  review  to  an  acute  disease 
state  and  a  preventive  or  health  maintenance  service) 

2.  To  develop,  implement  and  evaluate  a  continuing  education  program 
for  providers  in  relation  to  the  criteria  established  for  effective 
management  of  the  conditions  selected  for  study 

a.  To  develop  educational  materials  related  to  the  selected  condi- 
tions for  both  providers  and  consumers 

b.  To  hold  individual  and  group  educational  sessions 

To  accomplish  these  goals,  the  documentation  outlines  several  tasks  that 
were  to  become  part  of  the  operational  program: 

1.  Identify  most  prevalent  acute  disease  entities  and  preventive  or  health 
maintenance  service 

2.  Review  available  literature  pertaining  to  quality  review 

3.  Observe  quality  review  protocols  used  in  existing  programs 

4.  Present  appropriate  protocols  to  NVHC  providers  and  enlist  their  help 
in  selection  and  adaptation  of  protocols  to  NVHC  setting 

5.  Develop  audit  tools  to  evaluate  quality  of  care  delivered  to  NVHC  pa- 
tients relevant  to  the  selected  review  entities 

6.  Implement  audit  tools  and  protocols 

7.  Develop  check-list  and  guideline  sheet  for  review  entities 

8.  Replace  conventional  medical  record  progress  notes  with  check-list 
and  guidel ine  sheet 

9.  Construct  summary  sheets  to  identify  roles  and  educational  needs  of 
different  health  provider  levels  for  review  entities 

10.    Develop  patient  instruction  sheets  to  facilitate  patient  cooperation 
thereby  increasing  criteria  compliance  including  inservice  training 
of  NVHC  staff  in  use  of  sheets 


OPERATIONAL  ACTIVITIES 

The  documentation  provided  by  NVHC  provides  information  on  the  proposed 
quality  program  elements  and  the  implementation  of  those  elements  in  an  opera- 
tional setting. 
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The  following  QASAC  activities  have  been  implemented  or  accomplished  as 
of  March  2,  1976: 

1.  Development  and  implementation  of  QASAC  program  elements 

2.  Development  of  continuing  education  program 

QASAC  staff  reviewed  literature  pertinent  to  quality  review  including  proto- 
col development,  utilization  review,  medical  audit  and  other  evaluation  techniques. 
An  appropriate  bibliography  was  prepared  in  consultation  with  the  UCLA  Ambulatory 
Care  Evaluation  Project. 

Staff  then  reviewed  protocols  for  upper  respiratory  infection  (URI)  and 
well  child  care  that  had  been  implemented  in  other  quality  assurance  programs, 
specifically: 

1.  Ambulatory  Care  Project  (Beth  Israel  Hospital,  Lincoln  Laboratory,  MIT, 
affiliated  with  Harvard  Medical  School)  for  acute  and  chronic  condi- 
tions 

2.  Frontier  Nursing  Service,  Kentucky 

3.  PROMIS  Laboratory,  Department  of  Medicine,  Dartmouth  Medical  School 
for  URI 

4.  Department  of  the  Army,  AMOSIS  Manual 

5.  American  Academy  of  Pediatrics,  Standards  of  Child  Health  Care  and 
Criteria  for  Pediatric  Management 

After  review  of  these  protocols  and  consultation  with  experts  in  the  quality 
assurance  field,  QASAC  staff  decided  to  use  existing  protocols  as  a  starting 
point  for  meeting  the  needs  of  NVHC  patients,  providers,  and  the  practice  setting, 
rather  than  to  develop  entirely  new  protocols. 

Project  staff  then  participated  in  a  workshop  ("Educational  Patient  Care 
Audit  Workshop")  sponsored  by  the  California  Medical  Association  (CMA)  and  the 
California  Hospital  Association  (CHA)  to  familiarize  themselves  with  techniques 
necessary  for  the  development  and  implementation  of  medical  criteria.    A  follow- 
up  workshop  was  conducted  at  NVHC  by  a  CMA  faculty  representative,  to  familiarize 
staff  with  audit  methodology.    On  the  basis  of  the  CMA-CHA  workshops,  the  quality 
assurance  staff  chose  to  use  explicit  criteria  for  the  disease  conditions  chosen 
for  audit. 

Once  QASAC  staff  decided  on  specific  program  elements,  they  began  to  design 
specific  system  components  to  collect  and  compile  information.    The  first  step 
was  to  design  a  scheme  for  efficient  computer  access  to  certain  groups  of  medical 
records  (e.g.,  all  records  for  hypertensive  patients)  or  to  specific  records.  A 
computer  file  of  charts  containing  all  patient  encounters,  including  patient  iden- 
tification, birth  date,  provider  identification,  encounter  date  and  ICDA  diagnostic 
code,  was  used  to  access  all  records.    A  computer  program  was  developed  by  the 
director  to  select  encounters  containing  ICDA  codes  for  specific  conditions  chosen 
for  review  and  to  display  the  needed  information  in  a  report  format;  this  listing 
was  used  to  select  charts  for  audit. 

To  select  acute  disease  conditions  and  preventive-maintenance  services 
for  study  and  review,  information  was  gathered  on  disease  conditions  to  determine 
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the  most  frequently  occurring  conditions.    A  random  sample  of  200  (5%)  medical 
records  was  selected.    QASAC  staff  abstracted  diagnostic  data  from  the  charts, 
and  in  fact,  URI  and  well  child  care  were  the  most  commonly  encountered  chart 
entries.    To  validate  sample  results,  QASAC  staff  prepared  a  computer  summary 
of  all  medical  encounters  between  October,  1973,  and  October,  1974.    This  sum- 
mary confirmed  results  of  the  random  sample,  and  URI  and  well  child  care  were 
selected  as  review  diagnoses. 

Once  the  conditions  for  audit  were  confirmed,  the  QASAC  staff's  initially 
developed  criteria  were  revised.    Criteria  revision  took  place  in  a  series  of 
Peer  Review  Committee  meetings  of  the  family  health  teams  (physicians,  family 
nurse  practitioners,  clinical  pharmacists,  public  health  nurses,  social  workers), 
clinic  administration  personnel  (administrator,  medical  record  room  personnel) 
and  the  quality  assurance  grant  staff.    After  a  number  of  meetings,  a  consensus 
was  reached  regarding  the  acceptability  of  criteria  for  NVHC.    QASAC  staff  pre- 
sented developed  criteria  for  discussion  and  providers  supplied  medical  expertise 
needed  for  criteria  revision.    These  weekly  meetings  began  in  April,  1975. 

Resulting  criteria  were  primarily  process-oriented  and  were  associated  with 
information  that  could  be  obtained  from  medical  records.    In  addition,  criteria 
contained  objective  measurements  and  follow-up  activities  to  evaluate  overall 
quality  of  care.    Desired  performance  levels  for  criteria  were  set  at  levels 
of  compliance  considered  most  desirable  by  a  consensus  of  providers. 

Charts  containing  selected  encounters  were  pulled  from  the  medical  records 
library.    Pertinent  review  data  were  abstracted  from  charts  using  a  standard  form 
developed  by  QASAC  staff.    All  abstracts  were  manually  verified  by  qualified 
medical  personnel  before  becoming  available  to  peer  review  committee  members 
and  clinic  administrators. 

Based  on  study  staff  review  of  Patient  Care  Audit  Data  Display,  the  QASAC 
staff  presented  audit  results  for  each  reviewed  diagnosis,  outlining  the  purpose 
of  each  audit.    The  display  notes  specific  criteria  for  expected  performance  levels 
and  actual  performance  levels. 

Since  NVHC  employs  a  version  of  the  problem  oriented  medical  record  (POMR), 
each  topic  of  audit  (e.g.,  otitis  media,  hypertension)  has  specific  criteria  for 
the  Subjective,  Objective,  Assessment  and  Plan  (SOAP)  portions  of  the  POMR.  For 
each  of  these  data  areas,  there  could  be  one  or  many  criteria.    The  expected 
performance  level  is  the  percentage  of  medical  records  which  should  have  a  specific 
criterion,  so  each  criterion  has  a  corresponding  performance  level.    This  level 
is  established  by  QASAC  staff  in  conjunction  with  NVHC  providers.    The  actual 
performance  level  for  each  criterion  is  the  number  of  charts  meeting  a  criterion, 
divided  by  the  total  number  of  medical  records  reviewed. 

QASAC  staff  then  performed  statistical  analysis  of  chart  data  including 
comparison  with  desired  or  expected  levels  set  by  providers.    Criteria  and  per- 
formance levels  were  presented  to  providers  for  review  in  a  subsequent  peer 
review  meeting.    The  performance  levels,  as  noted,  were  percentage  figures  set 
by  providers  to  establish  an  acceptable  level  for  expected  performance.  The 
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documentation  noted  that  individual  non-compliant  providers  were  identified,  but 
not  publicly  mentioned  in  peer  review  meetings;  remedial  action  was  recommended 
to  correct  deficiencies. 

Once  the  audit  had  been  completed  for  a  particular  diagnosis,  results  were 
discussed  with  provider  staff.    The  discussions  centered  on  five  areas  of  health 
care: 

1.  Initial  assessment  of  patient  health  status 

2.  Physician  exam  and  history  entries 

3.  Diagnostic  work-up  for  specific  diseases 

4.  Review  of  pathology  reports,  pre-  and  postoperative  diagnosis,  admis- 
sion and  discharge  diagnosis 

5.  Diagnostic  information  including  laboratory  and  radiological  procedures 
to  review  validity  and  accuracy 

Reliability  of  audit  is  completely  dependent  on  quality  of  charts.    The  data 
presented  to  providers  were  aggregate  percentages  for  performance  levels. 

The  documentation  noted  the  following  possible  remedial  actions: 

1.  Explicit  delineation  of  responsibility  for  completion  of  certain  proce- 
dures, examinations,  history,  treatment  and  patient  education 

2.  Institution  of  provider  continuing  education  for  a  specific  area  of 
patient  care 

3.  Development  of  guidelines  for  use  by  providers  when  a  patient  presents 
certain  symptoms  or  complaints 

4.  Inservice  education  programs  for  instruction  of  staff  to  correct  identi- 
fied deficiencies  in  patient  care 

5.  Administrative  changes  (e.g.,  change  of  clinic  hours,  purchase  of  pedia- 
tric blood  pressure  cuffs) 

6.  Development  of  patient  education  materials  (e.g.,  URI  instruction  sheets) 
for  use  by  patients  with  certain  illnesses 

At  no  time  has  any  punitive  remedial  action  been  instituted  against  a  pro- 
vider; remedial  actions  are  intended  both  to  directly  improve  the  quality  of 
care  delivered  to  patients,  and  to  indicate  education  needs  of  providers. 

The  last  stage  of  quality  of  care  evaluation  was  to  perform  a  reaudit  of 
selected  charts  to  determine  efficacy  of  remedial  action.    Reaudit  has  been  per- 
formed for  well  child  care  and  URI  and  provider  performance  has  improved  in  most 
areas.    The  documentation  noted  the  cyclical  audit  method  is  necessary  for  con- 
tinuing reevaluation  of  criteria  and  performance  related  to  quality  care. 

Once  the  well  child  care  and  URI  were  completed,  QASAC  staff  selected  acute 
suppurative  otitis  media,  diabetes  and  urinary  tract  infection  for  audit.  The 
October  1  to  December  31,  1975,  Progress  Report  indicated  criteria  and  perfor- 
mance levels  were  established  for  diabetes  and  otitis  media.    The  QASAC  staff 
audited  records  and  presented  results  on  these  two  conditions  to  provider  staff. 
Peer  review  meetings  were  held  to  discuss  remedial  activities  for  diabetes.  Per- 
formance was  at  a  high  enough  level  for  otitis  media  that  remedial  action  was 
not  necessary. 
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Other  accomplished  tasks  related  to  development,  implementation  and  evalua- 
tion of  educational  materials.    Protocols  and  guideline-checklist  sheets  have 
been  developed  for  those  disease  conditions  audited  and  were  reviewed  for  educa- 
tional purposes.    Guideline  sheets  were  developed  by  revising  and  aggregating 
relevant  materials  from  previously  developed  protocols  and  progress  notes  from 
audited  records.    Some  providers  are  using  "Guideline  Sheets"  in  diagnosis  and 
treatment  of  URI  and  well  child  care. 

Inservice  education  has  been  implemented  to  increase  physician  and  nurse 
provider  awareness  of  criteria  and  provider  roles  in  management  of  selected 
services . 

Patient  education  materials  (instruction  sheets)  have  also  been  developed 
to  increase  patient  awareness  of  the  patient  role  in  proper  health  management. 
Educational  materials  are  available  in  both  English  and  Spanish  and  providers 
have  received  inservice  training  for  use  of  patient  education  materials. 


ADDITIONAL  COMMENTS 

Future  plans  of  QASAC  staff  include  training  medical  records  personnel  to 
abstract  medical  charts  for  quality  review.    Guideline  sheets  may  be  substituted 
for  progress  notes  currently  used  in  charts.    Both  activities  would  facilitate 
and  expedite  data  collection  for  quality  review. 

The  October  1  to  December  31,  1975,  Progress  Report  indicated  QASAC  staff 
would  continue  development  of  a  computer  program  which  will  link  hospitalization, 
ambulatory  and  registration  information  into  one  computer  file.    Presently,  infor- 
mation in  these  three  areas  is  contained  in  three  separate  files. 

Further  activities  include  audit  of  urinary  tract  infections  and  continued 
implementation  of  consumer  and  provider  educational  activities. 

NVHC  personnel  noted  several  unique  characteristics  in  the  NVHC  system. 
These  are  described  in  relation  to  their  importance  in  the  development  and  im- 
plementation of  NVHC  quality  review,  including: 

1.  Autonomous  role  of  director  of  Quality  Review  and  Utilization  Control 
has  facilitated  development  of  QASAC  unhindered  by  pressures  from  other 
interests. 

2.  Provider  staff  of  NVHC  is  young,  resulting  in  a  high  physician  turn- 
over and  absence  of  older  staff  for  guidance  and  support. 

3.  High  physician  turnover  has  made  it  difficult  for  QASAC  to  obtain  con- 
sistent support  for  quality  review. 

4.  NVHC  quality  review  program  is  jointly  funded  by  NVHC  and  CRMP.  Reduced 
CRMP  funding  has  limited  development  of  QASAC.    In  addition,  CRMP  has 
announced  an  eventual  complete  withdrawal  of  funds,  leaving  future  funding 
of  NVHC  QASAC  to  be  assumed  completely  by  the  corporation. 

5.  Physician  progress  notes  in  medical  records  are  generally  illegible  and 
are  a  serious  limitation  to  quality  review.    NVHC  hopes  medical  record 
notes  will  be  dictated  by  providers  and  transcribed  to  charts  in  the 
future. 


157 


QASAC  is  a  research  program  implemented  in  an  operational  health  maintenance 
setting.    Operational  activities  of  the  program  vary  with  the  amount  of  time 
staff  devotes  to  the  audit  activities.    QASAC  staff  presented  only  aggregate  data 
on  critical  performance  levels;  no  individual  noncompliant  providers  have  been 
singled  out. 

The  QASAC  staff  has  designed  an  evaluation  plan  for  the  program,  though, 
at  the  time  of  site  visit,  the  results  of  the  evaluation  were  unavailable.  The 
evaluation  may  now  be  available  since  the  CRMP  contract  required  an  evaluation 
be  completed. 
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SECTION  III 


DELIVERY      SYSTEM  CHARACTERISTICS 


NVHC  operates  as  a  publicly  supported  nonprofit  organization.  All  health 
care  is  provided  by  corporate  physicians  or  by  private  physicians  on  referral 
contracts  with  NVHC. 

NVHC  operates  primarily  as  a  prepaid  health  plan.    Ninety-five  percent  of 
revenue  is  received  from  either  Title  XIX,  DHEW,  or  private  payers  on  a  prepaid 
monthly  capitation  schedule;  5%  is  fee-for-service.    The  NVHC  service  population 
is  approximately  68%  white,  23%  chicano,  7%  black  and  2%  other. 

NVHC  has  categorized  its  services  into  seven  areas: 

1.  Professional  outpatient  services:    clinical  laboratory,  X-ray,  audiology, 
emergency  room,  social  services,  minor  surgery,  oral  surgery 

2.  Preventive  health  services:    periodic  health  review,  health  education 

3.  Professional  inpatient  services:    adult  and  pediatric  medical  and  sur- 
gical care,  postoperative  care,  obstetrics,  etc. 

4.  Hospital  services:    up  to  365  days  hospitalization,  diagnostic  and  re- 
habilitation services,  extended  care  services 

5.  General  nursing  services 

6.  Pharmaceutical  services 

7.  Appliance,  equipment  and  supplies 

NVHC  is  directed  by  a  24-member  community-based  board  of  directors.  Their 
decisions  are  implemented  by  an  executive  director  who  is  responsible  for  all 
NVHC  operations.    Directly  responsible  to  the  executive  director  are  deputy 
directors  of  health  services,  operations,  marketing,  and  planning  and  develop- 
ment, and  the  director  of  quality  review  and  utilization  control.    The  executive 
director  is  responsible  for  supervision  of  medical  directors  at  the  clinics. 

The  Deputy  Director  of  Operations  is  responsible  for  financial  management 
(accounting,  claims,  purchasing)  eligibility  verification  and  supervision  of 
some  nonmedical  personnel.    The  Deputy  Director  of  Planning  and  Development  is 
in  charge  of  facilities  planning,  special  projects,  and  liaison  with  governmental 
agencies.    Each  clinic  site  has  an  administrator  who  is  responsible  for  all  an- 
cillary (not  health  team)  services  and  personnel.    Known  professional  affiliations 
of  NVHC  are  with  UCLA  and  CRMP. 

Computer  hardware  used  at  NVHC  for  quality  assurance  is  located  at  UCLA. 
Software  available  at  UCLA  includes  SPSS  and  BIOMED  (statistical  packages)  and 
a  disease  index  program.    The  Problem  Oriented  Medical  Record  (POMR)  and  a  compu- 
terized encounter  form  are  used  for  documentation  and  subsequent  analysis  of 
ambulatory  encounters. 

Each  corporate-owned  health  center  has  the  capability  of  handling  approxi- 
mately 10,000  enrollees.  There  were  approximately  7,500  enrollees  as  of  March, 
1976.    The  primary  care  physician  to  population  ratio  is  1:1,200.    There  are 
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approximately  85  panel  specialists,  six  contracted  community  hospitals,  two  con- 
tracted extended  care  facilities,  in-house  dental,  lab,  X-ray,  pharmacy,  and 
social  services.    There  are  agreements  with  UCLA,  Cedars-Sinai,  Children's  and 
USC  medical  centers  to  provide  tertiary  specialty  and  inpatient  services.  NVHC 
contracts  with  some  valley  community  medical  groups  for  primary  care  services 
for  those  enrol  lees  not  living  near  corporate-owned  facilities. 
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SECTION  IV 

HCMS     STUDY  INFORMATION 


NVHC  agreed  to  participate  in  this  study  after  a  meeting  between  study 
staff  and  the  administrator  of  the  San  Fernando  site  (October  30,  1975).  During 
this  meeting,  HCMS  staff  explained  participation  requirements  and  arranged  for 
a  complete  follow-up  site  visit  on  January  16,  1976.    During  the  site  visit, 
the  data  collection  instrument  and  administrative  constraints  questionnaire 
were  used  as  interview  guides. 

The  following  documentation  has  been  received  from  NVHC: 

1.  Annual  Report,  October  1,  1974,  to  June  30,  1975.    A  Quality  Assurance 
System  for  Ambulatory  Care  (QASAC),  presented  to  California  Regional  Medi- 
cal Program  (CRMP),  includes  description  of  current  status,  history, 
objectives  and  activities  for  the  report  period,  conclusions  and  recom- 
mendations for  future  action,  audits,  results  of  upper  respiratory 
infection  (URI)  and  well  child  care  audit 

2.  Quarterly  Progress  Report  -  October  1  to  December  31,  1975.    A  Quality 
Assurance  System  for  Ambulatory  Care  (QASAC)  presented  to  California 
Regional  Medical  Program.    Includes  description  of  current  status  and 
future  plans  and  patient  care  audit  results  for  acute  suppurative  otitis 
media 

3.  Organizational  chart  of  San  Fernando  clinic 

4.  Medical  Encounter  Summary  form 

5.  Inpatient  Length  of  Stay  Authorization  form 

6.  Patient  Registration  form 

7.  Registration  form  computer  layout 

8.  Audit  sheets  for  upper  respiratory  infection  and  otitis  media 

This  documentation  provided  HCMS  staff  with  descriptive  information  on  the 
initial  program  development  and  activities  required  to  meet  contract  obligations 
to  CRMP.    It  described  specific  audits,  the  presentation  of  those  audits  to 
health  team  staff  members  and  the  audit  and  program  objectives  developed  during 
the  year  (October  1974  to  October  1975).    Development  of  criteria  for  specific 
acute  conditions,  the  medical  record  auditing  process  conducted  by  QASAC  staff 
and  the  development  of  computer  programs  for  QASAC  were  also  documented. 

Two  separate  site  visits  were  conducted  by  HCMS  staff.    The  first  interviews 
included  discussions  with  the  following  personnel: 

1.  Director,  Quality  Review  and  Utilization  Control 

2.  Administrator,  San  Fernando  Clinic  (NVHC) 

3.  Registered  Nurse,  Research  Associate,  Quality  Review  and  Utilization 
Control  Program. 

The  second  visit  involved  a  three-hour  meeting  with  the  director  which  covered 
specific  operational  activities  of  the  QASAC.    Total  interview  time  amounted  to 
over  seven  hours,  but  no  interviews  were  audio  recorded.    Topics  covered  included: 
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1.  Staffing  for  QASAC 

2.  Development  and  history  of  the  QASAC 

3.  Limitations  to  QASAC  posed  by  NVHC  practice  setting 

4.  Operational  activities  between  QASAC  staff  and  medical  staff 

5.  Medical  services  provided  by  NVHC  and  reviewed  in  QASAC 

6.  Administrative  structure  of  NVHC  and  clinic  sites 

7.  Computer  programs  developed  by  QASAC  staff  for  use  in  program 

8.  Specific  audit  procedures  used  in  QASAC 

9.  Activities  of  health  teams  in  QASAC;  interaction  between  health  teams 
and  QASAC  staff  within  meetings  where  audit  results  are  presented 

10.  Effectiveness  of  NVHC  quality  assurance  activities 

11.  Relationship  between  NVHC  corporate  structure  and  San  Fernando  clinic 


162 


DOWNSTATE  MEDICAL  CENTER 
Brooklyn,  New  York 


SECTION  I 
INTRODUCTION 


Downstate  Medical  Center  (DMC)  is  a  health  service  delivery  system  affiliated 
with  the  State  University  of  New  York.    The  Center  provides  teaching  and  residency 
services  to  the  university  medical  school.    Within  the  medical  center,  King's 
County  Hospital  is  the  major  outpatient-inpatient  facility  serving  population 
in  the  north  central  area  of  Brooklyn,  New  York.    The  Pediatric  Ambulatory  Ser- 
vice (PAS)  is  a  major  outpatient  department  of  King's  County  Hospital  (KCH), 
accounting  for  90%  of  all  outpatient  visits.    The  focus  of  investigation  was 
PAS  and  the  PAS  peer  review  system. 

KCH  operates  under  the  aegis  of  the  New  York  City  Health  and  Hospitals  Cor- 
poration (HHC),  a  quasi-independent  public  benefit  corporation.    The  HHC  Board 
of  Directors  is  a  legally  constituted  board  governing  KCH  and  17  other  New  York 
municipal  hospitals. 

PAS  was  included  in  the  study  because  it  represents  a  typical  urban  medical 
center  outpatient  facility  serving  an  inner  city  low-income  minority  population, 
with  450  patient  visits  daily.    Care  is  provided  around  the  clock  seven  days  a 
week. 

The  PAS  peer  review  quality  assurance  program  emphasizes  process  review  by 
audit  of  medical  record  content,  by  identification  and  selection  of  most  common 
diagnoses,  and  by  development  of  explicit  criteria  for  process  audit.    A  publica- 
tion describing  the  PAS  research  and  administrative  program  and  the  professional 
audits  of  medical  records  led  to  its  inclusion  in  the  study.    In  particular,  the 
program  is  important  because  of  its  effort  to  measure  the  amount  of  time  required 
to  perform  medical  record  audits. 
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SECTION  II 


QUALITY     ASSURANCE  PROGRAM 


The  Pediatric  Ambulatory  Services  (PAS)  quality  assurance  program  operates 
under  a  key  premise:    that  medical  records  reflect  the  quality  of  medicine  prac- 
ticed by  a  single  physician  dealing  with  a  single  patient.    While  the  program 
emphasizes  a  review  process  that  uses  explicitly  defined  standards  to  audit 
medical  record  contents,  there  is  also  strong  emphasis  on  the  necessity  of  a 
dialogue  between  provider  and  auditor  to  allow  review  of  some  aspects  of  care 
not  reflected  in  charts. 

The  director  of  PAS  expressed  to  study  staff  a  strong  interest  in  conducting 
research  in  quality  assurance  methods,  while  at  the  same  time  implementing  admin- 
istrative procedures.    The  impetus  for  use  of  the  particular  process  audit  was 
to  determine  a  time-effective  manner  of  conducting  chart  audits  by  a  specific 
group  of  physicians.    The  director  further  noted  that  possible  governmental 
mandate  of  quality  assurance  procedures  is  another  key  force  behind  program  im- 
plementation. 

This  section  will  describe  two  phases  of  the  PAS  audit  program: 

1.  The  implementation  of  the  formal  process  audit  review  system 

2.  Continuation  of  process  audit  after  the  formal  study  audit  was  com- 
pleted 

Implementation  of  the  process  audit  review  study  began  in  1973  with  a  com- 
mittee consisting  of  five  PAS  pediatric  service  supervisors  working  under  the 
director  and  available  as  medical  consultants  to  the  project's  clerical  staff 
whenever  necessary.    Their  first  documented  activity  was  to  outline  procedures 
for  implementing  the  review  system  as  follows: 

1.  To  identify  through  retrospective  survey  common  diagnostic  conditions 
in  outpatient  medical  records 

2.  To  select  the  most  common  diagnostic  conditions  by  using  frequency 
data 

3.  To  develop  criteria  for  each  diagnostic  condition 

To  conduct  the  retrospective  survey,  record  room  personnel  retrieved  records 
of  children  registered  a  year  before  the  study  period.    The  year  was  divided 
into  four  quarters  so  charts  would  represent  seasonal  variations.    Every  hun- 
dredth record  was  selected  by  clerical  staff  resulting  in  a  total  of  560  records, 
representing  1%  of  the  total  number  of  registered  patients.    Diagnoses  were  then 
reviewed  and  tabulated  by  rate  of  occurrence. 

A  patient's  last  recorded  visit  was  used  in  the  selection  process  and  all 
secondary  diagnoses  were  excluded.  Twenty-six  diagnoses  were  chosen  after  the 
committee  excluded  diseases  considered  minor  in  terms  of  complication  rates  and 
physical  impairment. 
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Committee  members  then  divided  diagnoses  selected  among  themselves  to  develop 

criteria  they  felt  to  be  "most  important  for  diagnosis  and  management"  for  the 

study.    Once  preliminary  criteria  had  been  developed,  the  committee  met  to  dis- 
cuss them  and  reach  a  consensus  about  criteria  for  all  selected  diagnoses.  Criteria 
were  then  chosen  upon  agreement  of  three  out  of  five  committee  members.  After 

criteria  were  established,  the  total  clinical  staff  reviewed  and  commented  on 
criteria  in  two  phases: 

1.  Prior  to  the  implementation  of  the  study 

2.  At  the  time  of  the  audits 

The  director  also  said  criteria  were  adjusted  based  on  interaction  between 
the  auditors  and  the  director  or  with  the  committee  as  a  whole. 

Criteria  were  chosen  if  they  were: 

1.  Essential  to  the  diagnosis 

2.  Acceptable  to  the  majority  of  health  care  providers 

3.  Able  to  be  met  within  the  limitations  of  the  health  facility's  services 

4.  Applicable  to  the  majority  of  patients  in  any  diagnostic  category 

5.  Few  enough  in  number  to  enable  rapid  review 

The  majority  of  criteria  were  designed  to  be  met  during  a  patient's  initial 
visit,  but  auditors  could  not  consider  a  chart  deficient  if  criteria  could  only 
be  met  by  subsequent  visits.    More  physicians  were  not  included  in  the  criteria 
development  phase  because  of  the  ensuing  complexity  and  additional  demands  on 
time.    The  director  said  development  could  conceivably  be  accomplished  with  a 
clinical  staff  of  10  to  15  physicians,  "but  when  you  are  dealing  with  40  to  50 
people,  it  gets  to  be  an  extremely  difficult  task  to  try  to  assemble  everybody 
democratical ly. " 

Once  criteria  development  was  completed,  audits  were  conducted  using  four 
medical  record  and  process  elements  selected  by  the  committee: 

1.  The  medical  history 

2.  Physical  findings 

3.  Laboratory  findings 

4.  Therapeutic  management 

Audits  were  conducted  weekly  using  a  random  selection  of  25  active  charts,  repre- 
senting approximately  5%  of  the  total  patient  population,  or  every  twentieth 
chart  transaction  on  the  day  of  the  audit.    Audits  were  conducted  on  the  same 
day  the  physician-patient  encounter  occurred.    The  normal  procedure  consisted 
of  one  committee  member  examining  the  chart,  while  a  second  member  read  criteria 
and  noted  audit  results.    During  this  two-member  audit,  the  other  four  committee 
members  observed,  discussed  and  participated  in  preparation  of  notes  for  provi- 
ders with  noncompliant  charts.    All  tasks  rotated  among  committee  members  and, 
at  a  later  stage,  included  the  medical  record  librarian. 

While  performing  the  audit,  each  member  of  the  review  committee  was  timed. 
Results  of  the  timing  survey  were  as  follows: 
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1.  Two  reviewers  audited  a  single  chart  in  3Q  seconds.    A  single  reviewer 
audited  a  chart  in  58  seconds  (sample  of  60  charts). 

2.  Of  337  charts  selected  for  review,  25%  could  not  be  completely  reviewed 
for  the  following  reasons: 

a.  12.4%  were  grossly  incomplete:    Patient  left  before  seeing  provider 
or  provider  wrote  "inadequate"  or  "no  note". 

b.  10%  contained  diagnoses  for  which  no  criteria  had  been  written.- 

c.  2.7%  were  illegible. 

3.  Of  the  253  charts  reviewed,  47.4%  were  deficient  in  one  or  more  criteria. 

4.  Deficiencies  tended  to  occur  more  often  in  criteria  related  to  the  physi- 
cal examination  of  the  child  and  least  often  to  those  involving  therapeu- 
tic management. 

5.  There  was  a  general  correlation  between  the  frequency  with  which  a  par- 
ticular diagnosis  occurred  and  the  number  of  chart  deficiencies  found 
for  charts  having  that  diagnosis. 

6.  Improvement  of  physician  compliance  with  audit  criteria  over  a  five- 
month  period  could  not  be  shown  conclusively. 

7.  Much  of  the  noncompliance  rate  was  a  result  of  incomplete  data  recor- 
ding.   As  many  as  38.2%  of  the  charts  could  be  completely  corrected; 
9.1%  partially  corrected,  i.e.,  chart  noncompliance  was  the  result  of 
incomplete  recording  of  data.    On  contacting  the  responsible  provider, 
additional  data  were  obtained,  changing  the  chart  status  to  compliant. 

The  present  review  system  is  considered  an  administrative-educational  tool, 
allowing  the  director  to  get  a  general  view  of  the  type  of  care  delivered,  and 
to  know  exactly  what  the  status  of  care  is  for  that  portion  of  services  covered 
by  the  audit.    The  current  status  involves  two  members  of  the  original  committee 
who  conduct  reviews  on  approximately  20  medical  records  per  week.    Audits  are 
completed  as  a  routine  administrative  function  by  these  two  physicians.    No  for- 
mal meetings  are  held  to  conduct  audits.    Further,  no  medical  records  are  specially 
pulled  for  the  audit.    The  two  reviewing  physicians  will  audit  records  that  have 
been  pulled,  waiting  for  a  specific  physician-patient  encounter  to  be  completed 
on  a  particular  day  before  conducting  the  audit.    The  audit  will  be  completed 
right  after  the  encounter.    The  schedule  for  review  is  left  to  the  discretion 
of  the  two  reviewing  physicians. 

Reviewers  are  still  auditing  the  same  26  diagnoses  identified  in  the  formal 
study  phase  of  the  program.    The  focus  remains  on  audit  of  single  encounters 
(visits),  because  70  to  80%  of  PAS  visits  are  for  acute  conditions  with  no  appoint- 
ment.   Audit  results  are  documented  in  a  log  book,  which  notes  the  following  data 
elements  for  each  review: 

1 .  Physician's  name 

2.  Date  of  review 

3.  Chart  reviewable  or  not  reviewable 
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4.  Chart  deficiencies 

5.  Feedback  to  physician  or  no  feedback 

6.  Physician  reviewer 

An  informal  note  from  the  director's  office  is  written  to  a  provider  when 
an  unfavorable  audit  is  discovered.    The  number  of  medical  records  reviewed 
over  the  past  three  years  is: 

1.  1973-74  -  253 

2.  1974-75  -  25 

3.  1975-76  -  76 

4.  1976       -  81 


ADDITIONAL  COMMENTS 

Subsequent  to  completion  of  the  formal  audit  study,  audit  procedures 
underwent  some  changes: 

1.  The  program  is  now  conducted  informally,  without  systematic  review 
schedules  or  methologic  sampling 

2.  There  is  no  measurement  of  the  time  spent  actually  conducting 
audits 

3.  Fewer  members  of  the  supervisorial  clinical  staff  are  involved  in 
audit  activity 

4.  Results  are  documented  less  precisely 

PAS  indicated  there  are  many  advantages  to  the  audit  method  of  quality 
assurance: 

1.  The  audit  is  rapid  and  easy  to  perform 

2.  The  audit  focuses  on  the  major  diagnostic  conditions  of  the  facility 

3.  It  lends  itself  to  performance  evaluation  of  individual  providers 
rather  than  groups 

4.  The  informality  of  the  system  has  encouraged  provider  cooperation 

Along  with  the  advantages  of  the  particular  system,  several  problem  areas 
at  PAS  were  also  noted: 

1.  PAS  is  a  large-volume  ambulatory  care  facility.    Its  size  is  reflected 
in  the  size  of  the  medical  records,  complicating  any  record  review. 

2.  Many  of  the  professional  staff  are  on  clinical  rotation  in  teaching 
programs,  and  are  only  temporarily  at  PAS 

3.  PAS  is  an  acute  care  facility  serving  a  population  of  predominantly 
low-income  minority  families.    Patient  noncompliance  with  PAS  requested 
follow-up  is  very  high  (40%)  and  patients  are  generally  seen  for  acute 
disease  and  not  seen  again  unless  the  problem  becomes  chronic  or 
another  problem  has  arisen 

4.  Due  to  HHC  budgetary  constraints,  the  staff  size  has  stayed  the  same 
in  the  PAS  for  three  years.    The  utilization  rate  has  increased  and 
the  department  does  not  often  have  time  to  conduct  additional  or 
extra  audit  and  review  activities. 
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Future  expansion  of  the  PAS  quality  assurance  program  is  not  planned,  al- 
though the  director  noted  a  similar  system  is  being  developed  for  the  Adult 
Ambulatory  Services  of  DMC-KCH.    He  also  expressed  interest  in  the  development 
of  criteria  that  could  be  reviewed  at  a  clerical  level.    The  director  is  pre- 
sently preparing  a  grant  request  for  funds  to  finance  further  research. 
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SECTION  III 


DELIVERY     SYSTEM  CHARACTERISTICS 


PAS  provides  outpatient  and  emergency  care  to  450  patients  daily.    The  ser- 
vice area  lacks  private  practitioners;  over  the  past  two  decades  the  hospital 
has  become  the  major  source  of  outpatient  care  in  the  community.    Acute,  non- 
appointment  care  is  the  predominant  type  of  service.    The  following  services 
are  provided  by  PAS  and  its  affiliated  hospitals  (King's  County  Hospital  and 
Downstate  Medical  Center): 

-  Acute  and  emergency  care 

-  Pediatric  and  newborn  care 

-  Pediatric  surgical  services 

-  Surgical  subspecialty  services  (Vascular,  N euro- surgery,  ENT,  Ophthal- 
mology, Plastic,  Thoracic,  etc.) 

-  Obstetric  and  gynecological 

-  Psychiatric  and  neurological 

-  Psychological  evaluation 

-  Hearing  and  speech  evaluation  and  therapy 

-  Social  services 

-  Home  care,  visiting  nurse 

-  Dental  service 

-  Cleft  palate  center 

Population  Characteristics: 

Socioeconomic  -  35%  eligible  for  public  assistance 

65%  ineligible  for  public  assistance  but  largely  low-income 

Ethnic  -  70%  Black 

28%  Puerto  Rican 
2%  Other 

Age  -  Children  1  day  to  13  years 

Residence  -  North  Central  Brooklyn  90% 
Other  parts  of  borough  9% 
Miscellaneous  1% 

Patients  are  heavily  dependent  on  public  transportation,  especially  bus 
systems.    A  considerable  number  of  patients  have  to  take  two  or  more  buses  to 
reach  the  hospital.    Average  travel  distance  ranges  from  a  few  city  blocks  to 
five  or  six  miles.    Referral  services  tend  to  be  at  the  Center  with  the  excep- 
tion of  well-baby  health  stations  located  in  the  patient's  neighborhood. 

Clinical  revenue  for  the  support  of  PAS  is  distributed  in  the  following 

way; 

62%  -  Fee-for-service  or  out  of  pocket 
37%  -  Medicaid 
1%  -  Third  party 


169 


Out-of-pocket  services  are  essentially  provided  by  the  city  through  mone- 
tary grants  to  families.    Although  the  fee-for-service  percentage  is  highest, 
patients  are  billed  using  a  sliding  fee  scale  developed  in  cooperation  with  New 
York  City  Department  of  Social  Services.    The  maximum  all-inclusive  fee  per  visit 
is  $41.00. 

Total  funding  for  the  fiscal  year  1975  was  $101,000,000.    The  annual  cost 
of  the  medical  staff  is  $1,400,000.    Total  operational  costs,  which  include 
medical,  nursing  and  administrative  staff,  amounted  to  $2,693,100  (Pediatric 
Emergency  Room  -  $1,561,000;  Out-Patient  Department  -  $1,132,100). 

The  staff  of  the  Pediatric  Ambulatory  Service  (PAS)  consists  of  38  full- 
time  equivalent  administrative  and  clerical  personnel,  40  full-time  physicians 
and  25  to  30  medical  students,  nurses  and  interns  who  rotate  through  on  educa- 
tional programs. 

PAS  maintains  active  medical  records  for  50,000  enrollees.    There  are  175,000 
to  190,000  patient  encounters  per  year  distributed  as  follows: 

Percent  of  Total 
No.  Encounters  Encounters 


Pediatric  Medicine 


Acute  Care  Services 
Follow-up 

140,000 
2,000 

80 
1 

Special  Clinics 

4,650 

3 

Dental 

3,500 

2 

Subspecialty  CI inics 

15,000 

9 

Pediatric  Surgery 

6,000 

4 

Pediatric  services  are  available  seven  days  a  week,  24  hours  a  day.  Approxi- 
mately 58%  of  patients  arrive  at  PAS  between  8:00  a.m.  and  4:00  p.m.,  34%  between 
4:00  p.m.  and  midnight;  and  8%  between  midnight  and  8:00  a.m. 

Hospital  affairs  are  directed  at  the  corporate  level  by  the  KCH  Council  of 
Executive  Directors  and  Council  of  Community  Boards.    They  also  represent  KCH  to 
the  President  and  Board  of  Directors  of  Health  and  Hospital  Corporation. 

PAS  is  administered  and  directed  jointly  by  the  administrative  director, 
Outpatient  Services,  the  director  of  the  Pediatric  Outpatient  Department,  the 
director  of  Social  Services  and  the  director  of  Nursing,  all  of  KCH.    The  director 
of  the  Pediatric  Outpatient  Department  is  responsible  for  the  quality  of  care  pro- 
vided by  the  entire  group.    He  is  assisted  by  two  associate  directors. 

Five  senior  pediatric  physicians  share  in  supervision  of  emergency  services. 
In  addition,  one  senior  pediatrician  is  responsible  for  the  general  pediatric 
clinic  and  one  and  a  half  full  time  equivalents  are  in  charge  of  house  staff 
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training.    Between  25  and  30  pediatricians  are  assigned  solely  to  primary  care. 
PAS  is  also  staffed  by  four  ambulatory  pediatric  fellows,  nine  to  ten  pediatric 
family  practice  residents  and  interns,  four  to  six  medical  students,  and  seven 
to  eight  pediatric  nurse  practitioners  who  are  assigned  solely  to  primary  care. 

The  medical  record  used  at  PAS  is  a  nonautomated  chronological  chart.  In 
addition  to  patient  and  provider  identification,  the  chart  contains  records  of 
diagnosis,  treatment,  follow-up,  lab  tests,  X-rays,  ancillary  services,  medica- 
tion and  consultation  referrals.    Results  of  diagnostic  procedures  and  lab  or 
X-ray  studies  are  recorded  on  continuation  sheets  and  filed  chronologically  in 
the  chart. 


Identification  numbers  are  assigned  to  each  new  patient.    The  patient  identi- 
fication system  is  keyed  to  unit  record  number  and  utilizes  the  Soundex  Card 
filing-retrieval  mechanism.    The  chart  is  retrieved  for  all  visits  and  hand 
carried  to  the  treatment  area.    The  same  chart  is  used  for  inpatient  admission. 
All  clinical  services  and  some  ancillary  services  are  coded  to  facilitate 
identification  for  statistical  description  and  billing. 

PAS  is  housed  in  a  facility  of  17,000  square  feet.    The  facility  has  26 
examining  rooms,  nine  offices  (three  for  physicians),  a  three-room  dental 
suite,  an  office  for  social  services,  and  a  two-room  laboratory.    There  are  also 
conference,  hearing  and  speech  evaluation,  record,  surgical  and  medical  treatment, 
casting  and  holding  ward  rooms. 

The  following  is  a  categorized  list  of  major  equipment  on-site: 

Emergency  -  Cardiac  monitor  and  defibrillator 

-  EKG  machines 

-  Byrd  respirator 

-  Suction  machine 


Medical  -  Titmus  vision  tester 

-  Vaso  audiometer 

-  Croupettes 

Laboratory  -  Coulter  counter 

-  Centrifuges 

-  Microscopes 


Dental 


Dental  X-ray  machine 

Dental  chairs  with  complete  equipment 


Educational 


Video  tape  machine 
Projectors  -  movies  and  slide 
Tape  recorder 
Camera 
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SECTION  IV 


HCMS     STUDY  INFORMATION 


The  Pediatric  Ambulatory  Service  (PAS)  agreed  to  participate  in  the  study 
in  a  telephone  conversation  December,  1975,  and  follow-up  correspondence  dated 
February.  23,  1976.    Documents  compiled  for  PAS  include  the  following: 

1.  Log  sheet  for  the  Audit  Record  Book,  detailing  results  of  specific 
audits 

2.  Criteria  standards  developed  by  PAS  clinical  staff 

3.  Organizational  charts  and  administrative  material  describing  PAS 
structure 

4.  "A  Chart  Audit  Peer  Review  System  in  an  Ambulatory  Service," 
Russo,  et  al.  (Pediatrics,  August,  1975) 

5.  PAS  utilization  information 

These  documents  describe  some  specific  audit  procedures,  audit  results  and 
criteria  developed  for  specific  disease  entities.  A  significant  amount  of 
information  was  collected  during  our  interview  with  the  director  of  PAS  on 
March  1  ,  1976. 

The  following  topics  were  discussed  during  the  interview: 

1.  History  and  current  status  of  the  Review  Program 

2.  Development  of  criteria 

3.  Review  methods  and  techniques 

4.  Feedback  and  impact  of  the  review  system 

5.  Advantages  and  limitations  of  the  review  system 

6.  Distinguishing  characteristics 

7.  Funding  and  staff 

8.  Association  between  practice  setting  and  review  system 

9.  Expansion  potential  of  the  review  system 

HCMS  staff  took  additional  time  to  review  sample  records  used  in  chart  review 
and  found  them  substantial  in  size  and  difficult  to  retrieve.  Apparently 
auditors  have  been  able  to  deal  with  this  problem. 

The  interview  clarified  the  intricate  details  of  developing  and  implementing 
a  peer  review  system  for  a  large  and  heavily  utilized  outpatient  department.  The 
documentation,  though  limited,  when  viewed  in  conjunction  with  the  information 
collected  during  the  interview,  clearly  delineated  the  operational  aspects  of  the 
PAS  program. 
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COMPREHENSIVE  HEALTH  SERVICES  OF  DETROIT 
Detroit,  Michigan 


SECTION  I 
INTRODUCTION 


Comprehensive  Health  Services  of  Detroit,  Inc.  (CHSD)  was  formed  by  the 
Model  Neighborhood  Comprehensive  Health  Program  in  1972  as  a  nonprofit  cor- 
poration.   On  November  8,  1972,  CHSD  was  granted  a  preliminary  Certificate  of 
Authority  under  Michigan  State  Acts  108  and  109  of  the  Public  Acts  of  1939. 
CHSD  began  operating  early  in  1973  as  a  nonprofit  prepaid  health  plan.  It 
has  continuously  contracted  with  the  Michigan  Department  of  Social  Services 
(MDSS)  since  1973  to  provide  prepaid  comprehensive  health  services  to  those 
Medicaid  recipients  who  elected  to  join  the  plan.    CHSD  was  regulated  by  the 
Commissioner  of  Insurance  through  1973.    As  of  December  31,  1975,  it  became 
subject  to  the  new  Michigan  HMO  Act  264  and  is  currently  licensed  pursuant  to 
Act  264  while  continuing  to  provide  comprehensive  health  services  pursuant  to 
its  contract  with  MDSS. 

Concurrent  with  the  ongoing  operation,  a  planning  grant  application  was 
submitted  to  the  Department  of  Health,  Education  and  Welfare  in  April,  1975. 
DHEW  subsequently  awarded  planning  grant  funds  in  the  amount  of  $124,900.00 
to  CHSD.    Currently,  the  group  is  conducting  planning  activities  in  order  to 
become  a  federally  qualified  health  maintenance  organization. 

As  a  large  prepaid  health  care  system,  CHSD  contracts  with  the  New  Center 
Medical  Plaza  Group  (NCMPG)  to  provide  all  physician  services.  Additionally, 
CHSD  employs  health  center  ancillary  staff  and  administration  personnel  suf- 
ficient to  serve  the  enrolled  population.    CHSD  currently  serves  a  population 
of  over  27,000  individuals,  from  a  broad  area  of  Detroit,  mostly  under  Title 
XIX. 

The  Administrative  Committee  manages  daily  operations  and  is  comprised  of 
the  following  members:    the  President  of  the  New  Center  Medical  Plaza  Group 
and  Acting  Medical  Director  of  CHSD;  the  Executive  Vice  President  and  Director 
of  the  Division  of  Marketing  and  Community  Affairs;  the  Vice  President-Treasurer 
and  Director  of  the  Division  of  Business  and  Fiscal  Affairs;  the  Director  of 
the  Division  of  Planning  and  Program  Development  Affairs;  and  the  Acting  Director 
of  the  Division  of  Health  Center  Affairs.    The  Executive  Vice  President  and  the 
Acting  Medical  Director  are  members  of  the  Board  of  Trustees.    The  President- 
Chairman  of  the  Board  has  no  direct  involvement  in  the  daily  operations. 

CHSD  was  included  in  the  study  because  of  its  implemented  and  documented 
quality  assurance  activity,  and  because  of  its  conditional  status,  under  the 
HMO  Act  of  1973,  which  requires  further  development  of  formal  quality  review. 
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The  CHSD  quality  assurance  program 
performance  levels  as  a  review  standard, 
cal  records  (POMR)  for  completeness,  and 


(QAP)  includes  use  of  individual 
audit  of  the  problem  oriented  medi- 
an active  continuing  education  program. 
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SECTION  II 


QUALITY     ASSURANCE  PROGRAM 


The  discussion  of  CHSD's  quality  assurance  program  (QAP)  is  divided  into 
three  areas: 

1.  Program  described  in  documents  supplied  to  HCMS  staff 

2.  Operational  activities  oriented  toward  QAP  revision  of  review  system 

3.  Operational izati on  of  medical  audit  procedures 


BACKGROUND  AND  FORMALLY  PLANNED  ACTIVITIES 

The  quality  assurance  program  was  officially  implemented  in  June,  1975,  for 
two  reasons: 

1.  To  conform  the  CHSD  philosophy  of  quality  care  with  dignity  and  com- 
pliance to  contractual  obligations  to  MDSS 

2.  To  document  a  formal  quality  assurance  program  in  preparation  for 
federal  HMO  qualification 

In  documentation,  CHSD  proposed  to  develop  and  implement  an  internal, 
ambulatory  QAP  allowing  for  evaluation  of  both  in-hospital  and  ambulatory  care 
modes  of  health  delivery.    The  documentation  describes  four  major  goals  of  the 
QAP: 

1.  To  identify  the  educational  needs  of  the  physician 

2.  To  alter  patterns  of  health  care  to  conform  with  community  standards 

3.  To  provide  a  "feedback"  mechanism  to  adjust  staffing  patterns,  facili- 
ties, supplies,  equipment,  services,  etc. 

4.  To  improve  generally  the  acceptability  and  accessibility  of  high 
quality  health  care  to  the  patients 

CHSD  documents  note  that  "quality  assurance  program(s)  should  be  designed 
to  meet  specific  needs  of  a  particular  HMO"  and  that  specific  programs  can  be 
different,  but  QAP  results  should  be  consistent.    CHSD's  program  emphasizes  the 
importance  of  using  individual  performance  levels  to  judge  the  quality  of  care 
in  a  total  system.    Performance  levels  can  be  compared  with  national  data  to 
judge  quality  of  care,  using  the  following  parameters: 

1.  Number  of  patients  examined  and  treated  per  hour 

2.  The  amount  and  appropriateness  of  laboratory  and  X-ray  procedures 

3.  The  number  of  referrals  to  other  specialties 

4.  The  number  of  referrals  to  ancillary  professional  personnel  such  as 
nutritionists,  social  workers,  nurses,  etc. 

5.  The  use  of  a  formulary  and  generic  prescribing  habits 
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The  documentation  also  notes  two  general  areas  the  QAP  should  favor: 

1.  Auditing  the  completeness  of  the  medical  record  (use  of  POMR) 

2.  Documentation  of  continuing  education  activities  by  physicians 

The  Medical  Council  has  specific  responsibilities,  one  of  which  is  to 
evaluate  contracts  with  physician  service  groups  delivering  medical  services, 
and  to  document  the  quality  of  these  services.    The  cornerstone  of  the  QAP  is 
the  contractual  obligations  the  physician  service  corporation  has  for  conduc- 
ting certain  review  activities.    The  specific  QAP  responsibilities  of  the 
Medical  Group  are: 

1.  To  provide  audits  of  medical  records 

2.  To  develop  and  perform  a  quality  assurance  audit 

3.  To  receive  and  review  professional  activity  data,  problems,  grievances, 
and  recommended  corrective  measures 

The  Medical  Council,  composed  of  chiefs  of  medical  services  and  key  ad- 
ministrative personnel,  elects  its  own  Chairman  and  Vice-Chairman  and  develops 
its  own  rules  and  regulations  for  approval  by  the  CHSD  Board  of  Directors.  The 
Medical  Director  and  Executive  Vice  President  are  ex-officio  members  of  the 
Council,  participating  in  discussions  only.    Terms  for  members  last  six  months 
and  are  staggered  to  insure  continuity  of  philosophy  and  action.    The  Council 
meets  monthly. 

After  the  audits  are  completed  by  the  Quality  Assurance  and  Utilization 
Review  Committee,  the  Medical  Council  reviews  those  medical  records  where  a 
variance  of  standards  has  been  determined.    On  the  basis  of  the  audit  and  Council 
decision-making,  the  Council  recommends  corrective  action.    Subsequent  to  the 
corrective  action,  the  Audit  Review  Person  must  supply  the  Council  with  data  to 
determine  the  effectiveness  of  the  action. 

Finally,  the  Council  must  submit  a  written  report  to  the  Board  of  Direc- 
tors about  the  audit  program. 

The  last  area  of  quality  assurance  documentation  deals  with  the  activities 
of  the  Audit  Review  Person,  and  the  auditing  of  medical  records.    These  were 
carried  over  to  the  operational  program,  and  are  discussed  below. 


OPERATIONAL  ACTIVITIES 

Operational  activities  are  divided  into  two  areas:  (a)  quality  assurance 
system  revision,  and  (b)  implementation  of  audit  procedures. 


System  Revision 

Although  the  documentation  noted  no  effort  to  educate  physician  staff  to 
quality  assurance  methods,  the  Medical  Director  did  conduct  a  literature  review 
of  quality  assurance  material,  forwarding  certain  articles  to  medical  staff 
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members.    He  also  held  discussion  meetings  with  staff  members  to  begin  building 
support  for  an  audit  program. 

Other  operational  activities  preceding  systems  revision  included: 

1.  Meetings  with  clinical  staff  regarding  development  of  specific  quality 
assurance  program  elements 

2.  Establishment  of  a  Medical  Audit  Quality  Assurance  Committee 

Subsequent  to  these  organizational  activities  were: 

3.  Two  formal  audits  of  medical  records 

4.  First  Medical  Council  Meeting  (February) 

The  Medical  Audit  and  Quality  Assurance  Committee  (QAC)  was  organized  to 
concentrate  operational  activities  in  a  smaller  and  more  specific  administrative 
unit  (rather  than  the  Medical  Council).    This  committee  is  comprised  of  five 
physicians,  plus  the  Medical  Director,  the  Medical  Records  Librarian  and  the 
Director  of  the  Social  Work  Services  Department.    This  committee  convenes  once 
a  month,  usually  for  two  hours.    Its  duties  and  responsibilities  are  similar 
to  those  planned  for  the  Medical  Council  except  that  specific  recommendations 
for  corrective  action  can  be  implemented  readily  because  departmental  barriers 
are  removed. 

The  results  of  system  revision  reflect  expanded  committee  activities: 

1.  Receive  standards  set  by  department  and  other  services; 

2.  Receive  information  relative  to  problems  identified  by  departmental 
and  other  audit  committees,  or  by  consultants;  receive  complaints, 
etc.,  not  within  the  lines  of  the  department  identifying  suboptimal 
performance; 

3.  On  occasion,  institute  its  own  interdisciplinary,  interservice  com- 
prehensive review  for  identification  of  problems  overlooked  by  other 
methods  of  problem  identification; 

4.  Refer  identified  problems  received  or  discovered  to  the  appropriate 
department  or  departments  for  standard  setting  or  other  action; 

5.  Set  standards  which  involve  the  entire  delivery  system,  after  their 
ratification  by  medical  staff; 

6.  Determine  the  priority  of  standards  to  measure  compliance; 

7.  Affirm  or  modify  data  collection  proposals; 

8.  Affirm  or  modify  assignments  to  Audit  Review  Person  and  receive  reports 
from  Audit  Review  Person; 

9.  Through  normal  channels,  recommend  a  timetable  for  improvement  activi- 
ties by  the  appropriate  agent,  and  receive  confirmation  that  action 

has  occurred.    These  recommendations  for  attention  may  result  in  actions 
in  the  form  of  education,  system  changes,  purchasing,  or  economic  or 
professional  sanctions. 

10.  Make  results  of  progress  available  to  appropriate  executive  bodies; 

11.  Keep  medical  staff  informed  of  standards  and  progress  in  achieving 
them; 

12.  Select  date  for  next  study  or  follow-up  review  based  upon  the  recog- 
nized need  for  a  planned  cycle  of  study. 
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The  role  of  the  Medical  Council  in  quality  assurance  is  to  implement  recom- 
mendations from  the  Medical  Audit  Committee.    The  role  of  the  Council, consisting 
of  supervisory,  administrative  and  medical  personnel,  is  to  set  policy  for  quality 
assurance,  to  interact  with  other  administrative  committees  (e.g.,  Administrative 
Committee)  and  to  gauge  overall  performance  of  CHSD  as  reflected  in  data  from 
audits. 

Audit  decision-making  occurs  within  the  Medical -Audit  Committee.  The 
Medical  Director  is  extremely  active  in  all  facets  of  the  audit  program,  and  prepares 
reports  to  the  Medical  Council  on  specific  audits.    Discussions  about  the  impli- 
cations of  certain  audits  occur  within  the  Medical  Council.    Because  of  the 
developmental  nature  of  the  program,  the  Medical  Director  is  depending  less  on 
committee  actions  than  on  his  own  activity. 

The  Audit  Review  Person's  (ARP)  main  responsibility  is  to  conduct  the  in- 
patient utilization  monitoring,  inpatient  medical  care  evaluation  and  referral 
monitoring.    She  collected  data  for  all  cases  involved  in  the  hypertension  audit, 
and  also  conducted  the  audits.    The  ARP  indicates  she  conducted  the  audits  based 
on  instructions  from  the  Medical  Director  and  reported  results  directly  to  him. 

The  operational  system  revision  directly  affected  the  type  of  review  process 
implemented,  although  the  procedural  elements  of  the  audit  system  remained  as 
outlined  in  planning  documents.    The  Audit  Review  Process  (Plan)  consists  of 
five  steps  as  follows: 

Step  1:    Problem  Identified  by  Medical  Audit  -  QAC.    A  study  group  consists  of 
patients  having  some  common  characteristic.    The  group  must  be  defined  and  time 
period  must  be  chosen  so  a  reasonable  number  of  cases  can  be  studied.  Dates 
must  be  set  for  adoption  of  pattern  standards,  data  retrieval  and  completion 
of  study. 

Step  2:    Pattern  Standards  and  Monitor  Parameters  established  by  Chiefs  of 
Services.    A  pattern  standard  is  a  percentage  indicating  how  often  a  given  jus- 
tification of  outcome  or  process  value  occurs  per  100  patients  of  care  given. 
"Monitor  parameters"  are  those  selected  parameters  which  are  checked  at  regular 
intervals. 

Step  3:    Audit  Review  Person  reviews  medical  record  for  problem.    The  manner 
of  review  is  designated  by  the  Medical  Council.    Comparison  of  the  medical 
records  with  the  temporary  standard  disease  protocol  allows  the  Audit  Review 
Person  to  identify  on  a  preliminary  basis  records  which  may  fall  into  the 
following  categories: 

1 .  Superior  Care 

2.  Standard  Care 

3.  Inferior  Care 

4.  Deficiencies  due  to  special  circumstances,  e.g.,  unavailability  of 
certain  facilities,  etc. 

5.  Deficiencies  due  to  lack  of  knowledge 

6.  Deficiencies  due  to  incomplete  performance 
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Step  4:  Chart  sent  to  Medical  Audit  -  QAC  for  review  and  determination  of  com- 
pliance regarding  standard  of  care. 

Step  5:    Recommendations  are  made  by  Medical  Audit  -  QAC  and  sent  to  the  Medical 
Council  where  action  is  taken  at  the  Medical  Staff,  Administrative  Staff,  or 
Board  of  Directors  level. 

At  meetings  subsequent  to  the  submission  of  recommendations,  the  Audit 
Review  Person  provides  assessment  of  the  effectiveness  of  the  action  taken 
to  correct  the  identified  deficiencies  in  medical  care. 


Implemented  Audit  Procedures 

Two  quality  review  audits  were  conducted  at  CHSD.  The  first  was  conducted 
in  September,  1975,  to  evaluate  the  problem  of  hypertension  according  to  expli- 
cit and  implicit  criteria.  The  second  audit,  February,  1976,  evaluated  medical 
charts  using  implicit  criteria. 

The  decision  to  conduct  both  audits  was  made  by  the  Medical  Director.  The 
QAC  was  not  consulted  for  selection  of  the  review  topic.    The  Medical  Director 
indicated  his  responsibility  focuses  on  initially  developing  and  implementing 
the  QAP  which  requires  him  to  take  a  strong  role  in  the  entire  program. 

The  hypertension  audit  involved  establishing  criteria  for  the  treatment 
and  diagnosis  of  hypertension  patients.    A  total  sample  of  patients'  medical 
records  (150)  containing  the  diagnosis  hypertension  was  pulled  for  review  by 
the  Audit  Review  Person.    The  ARP  reviewed  the  charts  herself,  noting  the  number 
of  specif ic  cases  where  services  for  hypertension  were  not  provided.    The  ARP, 
an  RN,  indicated  she  employed  some  medical  judgment  in  the  review,  though  an 
explicit  outline  was  used.    The  following  data  were  reviewed:    age,  sex,  weight, 
height,  blood  pressure,  past  history;  plan  of  treatment;  physical  findings;  and 
lab  tests.    Certain  charts  were  not  forwarded  to  the  QAC  for  review,  but  went 
directly  to  the  Medical  Director  for  review.    On  the  basis  of  study  staff  review 
of  minutes  (September  16,  1975)  it  appears  some  blood  pressures  were  being  re- 
corded inaccurately. 

Feedback  consisted  of  a  memorandum  from  the  Medical  Director  to  the  two 
RNs  responsible  for  taking  blood  pressures  throughout  the  clinic.  Enclosed 
with  the  memorandum  was  the  standard  pattern  of  care  for  hypertension  as  adop- 
ted by  the  QAC.    No  resample  or  reaudit  was  conducted  to  determine  if  patterns 
of  care  had  changed  as  a  result  of  the  review. 

In  the  second  operational  audit,  medical  records  were  pulled  for  review 
by  the  committee  from  scheduled  appointments  between  the  hours  of  3:00  to  5:00 
p.m.,  February  13,  1976.    This  audit,  conducted  by  the  QAC,  required  committee 
physicians  to  make  a  comprehensive  assessment  of  the  total  medical  record  by 
commenting  on  any  aspect  of  the  medical  record  on  a  blank  sheet  of  paper.  Com- 
ments were  aggregated  for  review  and  several  suggestions  were  made,  for  example: 
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1.  To  the  medical  record  librarian  with  regard  to  alerting  physicians 
when  records  were  incomplete. 

2.  To  physicians  regarding  the  legible  signatures  and  suggestions  per- 
taining to  the  patterns  of  care  for  patients.    Apparently  no  follow-up 
was  done  to  determine  a  change  in  behavior  as  a  result  of  the  review. 

CHSD  personnel  noted  the  Michigan  State  audit  was  scheduled  to  take  place 
on  April  14,  1976,  to  accomplish  the  following  three  things: 

1.  Facility  Division  would  examine  the  structural  aspects  of  the  facility 

2.  Ancillary  Service  Contracts  were  to  be  reviewed,  approved  or  disapproved 

3.  A  random  sample  of  500  medical  records  was  to  be  reviewed  (eight  audit 
items) 

CHSD  personnel  view  these  external  audits  as  part  of  their  overall  quality  assur- 
ance effort. 


ADDITIONAL  COMMENTS 

The  CHSD  quality  assurance  program  is  in  its  initial  stage  of  implementation. 
Though  the  provided  documentation  presented  a  fairly  coherent  system,  actual 
operations  were  affected  by  many  administrative  factors.    It  is  difficult  to 
document  the  complexity  of  such  interaction  in  development  of  a  new  program. 

Unique  characteristics  or  problems  in  implementing  QAP  were  described  as 
follows: 

1.  Limited  amount  of  time  Medical  Director  and  physicians  on  the  Audit 
Committee  can  give  to  development  of  the  QAP 

2.  Accessibility  of  sources  available  for  commitment  to  the  QAP 

3.  Coordination  between  the  Audit  Coordinator,  Audit  Committee,  Medical 
Director,  Medical  Council,  Administrative  Committee  with  regard  to 
getting  institutional  commitment  for  a  formalized  QAP 

4.  Difficulty  in  getting  staff  physicians  together  for  peer  review 
meetings 

5.  Priorities  of  the  CHSD  delivery  system  are  oriented  toward  achieving 
quality  health  care  with  sensitivity  to  cost  containment  while  main- 
taining overall  fiscal  viability. 
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SECTION  III 

DELIVERY     SYSTEM  CHARACTERISTICS 


CHSD  is  comprised  of  two  distinct  organizational  entities:    The  Comprehen- 
sive Health  Services  of  Detroit  and  a  physician  provider  service  group  (New 
Center  Medical  Plaza  Group,  Inc.).    CHSD  contracts  with  the  physician  group 
to  provide  all  medical  services  to  a  defined  population.    Everyday  operations 
are  carried  out  by  an  Administrative  Committee  comprised  of  the  following:  the 
President  of  the  New  Center  Medical  Plaza  Group  and  Acting  Medical  Director  of 
CHSD;  the  Executive  Vice  President  and  Director  of  the  Division  of  Marketing  and 
Community  Affairs;  the  Vice  President-Treasurer  and  Director  of  Division  of  Business 
and  Fiscal  Affairs;  the  Director,  Division  of  Planning  and  Program  Development 
Affairs;  and  the  Acting  Director,  Division  of  Health  Center  Affairs.    They  have 
specific  functions  and  are  jointly  responsible  to  the  Board.    The  President- 
Chairman  of  the  Board  has  no  direct  involvement  in  daily  operations.  The 
Administrative  Committee  relates  directly  to  the  Board  of  Directors  and  imple- 
ments policy  mandated  by  Board  action.    The  Vice  Presidents  and  Medical  Director 
are  appointed  by  the  Board.    The  organizational  structure  complies  with  the 
Michigan  State  Law  relating  to  HMOs.    The  governing  body  comprises  nine  board 
members,  consisting  of  three  subscribers,  three  persons  from  the  business  com- 
munity, and  three  providers,  thus  one-third  of  the  governing  board  is  represented 
by  enrol  lees  of  CHSD. 

In  addition  to  the  prepaid  capitation  contract  there  is  also  a  minimal 
fee  for  service  income.    CHSD  has  derived  its  major  income  from  contracts  with 
government  agencies  and  a  grant  from  Hi  11 -Burton. 

The  services  provided  by  CHSD  include  the  following: 

1.  Complete  inpatient  hospital  and  emergency  room  services 

2.  Diagnostic,  treatment,  preventive  and  rehabilitative  physician  and 
nursing  services 

3.  Comprehensive  health  examinations  and  other  screening  tests 

4.  Immunizations  as  required  by  needs  of  enrolled  member 

5.  Complete  podiatric  services 

6.  Complete  family  planning  services 

7.  Laboratory  and  X-ray  services  including  diagnostic  and  therapeutic 
radiological  procedures 

8.  Pre-  and  post-natal  care 

9.  Eye  exams,  refractions  for  glasses  and  eye  glasses 

10.  Prescription  drugs  prescribed  by  CHSD  physicians 

11.  Emergency  ambulance  service  to  centers  and  other  affiliated  facilities 
when  ordered  by  authorized  center  staff 

12.  Pharmacological  services,  including  coordination  of  and  individualized 
counseling  service  to  members  for  drug  related  problems 
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13.  Health  education 

14.  Social  work  services 

15.  Rehabilitation  services,  including  physical,  occupational  and  speech 
therapy 

16.  Nutritionist,  dietetician  and  home  economist  services 

17.  Outpatient  visits  to  psychiatrists  upon  referral 

18.  Prosthetic   appliances  when  prescribed  by  CHSD  physicians 

19.  Basic  mental  health  services  providing  outpatient  evaluation;  crisis 
intervention  services;  and  consultation  and  individual  and  group  therapy 
limited  to  10  visits 

The  Physician  Provider  Service  Group  is  composed  of  15  FTE  physicians  in 
the  primary  care  specialties,  eight  consultants  in  the  various  specialties;  and 
two  physician's  assistants.    Other  specialties  are  on  a  session  or  referral 
basis.    When  those  specialties  on  a  dollar  volume  indicate  payment  approximates 
half  time,  arrangements  are  consummated  to  bring  that  specialty  in  house. 

In  addition  to  the  consultants  who  spend  sessions  at  either  one  or  both 
centers,  the  New  Center  Medical  Plaza  Group  arranges  for  a  consultant  panel  to 
see  those  members  who  have  a  highly  specialized  or  emergent  problem  that  cannot 
be  properly  handled  on  site.    The  specialists  covered  in  the  consultant  panel 
who  are  paid  on  a  fee  for  service  basis,  based  upon  established  screens,  are  as 
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Radiology 
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Urology 

5. 

Al lergy 

6. 

Pediatric  Allergy 

7. 

Cardiac  Surgery  (Pacemaker) 

8. 

Cardiology 

9. 

Cardiovascular  and  Thoracic  Surgery 

10. 

Child-Adolescent  Psychodiagnostic  and  Psychotherapeutic  Services 

n. 

Dermatology 

12. 

EENT 

13. 

Endocrinology 

14. 

Hand  Surgeons 

15. 

Gastroenterol ogi  st 

16. 

Hematologist 

17. 

Neurologist 

18. 

Neuro-Surgery 

19. 

Oncology 

20. 

Ophthalmology 

21. 

Oral  Surgery 

22. 

Proctology 

23. 

Physical  Medicine 

24. 

Pulmonary  Medicine 

25. 

Radiation  Therapy 

26. 

Rheumatology 
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The  distribution  of  physicians  by  specialty  is  as  follows: 


FTP 

Halt-Time     Consultant  (Session) 

Tntprnal  Medicine 

4 

4  1 

PpH  i  a t*  ri  r ^ 

4 

UCIICI  Q  1     JUI  y  C  1  Jr 

C 

Ootometrv 

2 

-  (Schaefer) 

OB-GYN 

3 

Podi  a try 

1 

ENT 

1 

Neurology 

1 

Orthopedic  Surgery* 

-  (Schaefer) 

Psychiatry 

1 

Radiology* 

-  (Schaefer) 

*It  should  be  noted  that  Orthopedic  Surgery  and  Radiology  services 
are  available  at  the  Schaefer  Center  and  not  the  New  Center  Plaza 
at  the  present  time.    It  is  important  to  note  members  are  referred 
where  practical  to  the  Schaefer  Clinic,  likewise  members  are  re- 
ferred to  the  Plaza  Center  for  those  services  not  available  at 
Schaefer. 

Approximately  5-6%  of  the  27,000  patients  are  Medicare  recipients.  The 
majority  of  this  population  is  from  the  center  city  area  of  Detroit. 

The  majority  of  the  CHSD  payment  for  services  comes  from  the  prepaid  capi- 
tations from  Medicaid.    A  very  limited  amount  of  payment  is  fee-for-service. 

In  addition  to  completing  the  medical  record  for  each  patient-physician 
encounter,  an  encounter  form  is  completed  for  one  reason:    statistical  informa- 
tion gathering. 

The  medical  records  contain  the  traditional  chronological  clinic  sheet. 
They  are  filed  by  terminal  digit  and  are  color-coded  for  different  categories. 
The  records  are  divided  into  four  sections: 

1.  Progress  notes 

2.  Prescription  sheets 

3.  Tests  -  lab,  X-ray,  EKG,  etc. 

4.  Patient  information  Outside  Reports 


PHYSICAL  LAYOUT 

The  New  Center  Plaza  Clinic  occupies  the  former  Gold  Key  Inn  where  a 
portion  of  its  65,000  square  feet  has  been  converted  into  a  health  center. 

The  Schaefer  Clinic,  located  in  Northwest  Detroit,  has  approximately 
7,000  square  feet  and  is  a  primary  care  satellite  unit  of  CHSD. 
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AFFILIATIONS 


CHSD  has  several  hospital  and  other  provider  affiliations  through  which 
services  are  provided. 

1.  Relationships  exist  with  seven  hospitals  within  the  CHSD  service 
area.    CHSD's  contracting  Medical  Group's  physicians  enjoy  staff 
and  admitting  privileges  within  at  least  one  of  the  affiliated  hos- 
pitals -  combined  they  have  privileges  at  all  affiliated  hospitals. 

2.  Affiliations  with  three  nursing  homes. 

3.  The  VNA  and  Homemaker  Services  of  Metropolitan  Detroit  provide  ser- 
vices for  CHSD  on  a  contractual  basis.    Public  Health  nurses  (employed 
by  CHSD)  also  make  home  visits. 

4.  Pharmaceutical  services  are  provided  by  33  community  pharmacies.  An 
on-site  pharmacy  is  in  the  developmental  stage. 

5.  Referral  sources  for  substance  abuse  and  alcoholism  treatments  are 
available  in  the  community  for  CHSD  enrol  lees.    The  Henry  Ford  Hospital 
Pallister  Clinic  is  one  of  these  resources. 

6.  Extensive  interplay  with  various  governmental  agencies. 

7.  Michigan  State  Department  of  Social  Services. 

8.  Detroit  Department  of  Health. 

9.  American  Arbitration  Association. 


LIMITATIONS  OF  THE  DELIVERY  SYSTEM 

1.  Problems  encountered  by  CHSD  with  regard  to  the  financial  status. 
Medicaid  reimbursement  has  dictated  how  much  service  will  be  pro- 
vided to  a  specific  population  within  a  specific  time  period. 

2.  The  turnover  rate  of  the  medical  staff.    Although  it  is  not  high  it 
is  quite  unpredictable.    Dr.  Batchelor  was  forced  to  recruit  several 
foreign  medical  graduates  to  stabilize  the  provider  group.    The  prob- 
lem appears  stable  at  this  time. 

3.  The  Plaza  Center  was  formerly  a  motel  that  was  converted  into  a  medical 
center.    Delivery  of  service  did  not  require  utilization  of  all  the 
available  space;  excess  space  was  leased  to  Henry  Ford  Hospital  on  a 
short  term  basis. 
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SECTION  IV 


HCMS     STUDY  INFORMATION 


Comprehensive  Health  Services  of  Detroit  was  considered  for  the  study 
when  HCMS  staff  discovered  their  application  for  an  HMO  Grant  and  documentation 
therein  regarding  implementation  of  some  quality  assessment  activities.  CHSD 
agreed  to  participate  in  a  study  during  a  telephone  discussion  on  December  8, 
1975.    HCMS  reviewed  and  compiled  initial  documentation  on  January  27,  1976. 

The  documentation  received  prior  to  the  site  visit  of  February  19,  1976, 
is  as  follows: 

1.  Developmental  Grant  Application  to  become  a  qualified  HMO  (March  31, 
1975),  600  pages 

2.  Minutes  for  Quality  Assurance  Committee  (September  26,  1975); 

3.  Medical  Record  Audit  Sheet 

4.  Responsibilities  and  Procedures  for  a  Medical  Audit  Procedure 

5.  Full  charge  to  perform  retrospective  study  of  hospital  utilization 

6.  Chart  on  length  of  stay  and  QA 

7.  Medical  Record  Audit  Committee,  appointed  members  and  ex-officio 
members 

8.  Duties  and  responsibilities  of  the  Medical  Record  Audit  and  Quality 
Assurance  Committee 

9.  Medical  Council  Membership 

10.  Technical  service  form;  the  number  of  hours  worked  per  week  by  physi- 
cians 

11.  Total  number  of  patients  scheduled,  walk-ins,  no  shows,  cancellations 
and  the  number  of  patients  actually  seen  by  physicians  on  a  weekly 
basis 

12.  Staff  roster  for  CHSD 

The  documentation  described  the  program  well,  but  was  somewhat  dated,  and 
lengthy.    At  times  it  was  difficult  to  distinguish  the  precise  and  critical 
points  of  the  development.    More  information  pertaining  to  current  and  opera- 
tional aspects  of  the  CHSD-QAP  would  be  useful. 

The  site  visit  was  conducted  by  HCMS  staff  on  February  19,  1976,  and  in- 
cluded interviews  with  the  following  personnel: 

1.  Executive  Vice  President  and  Director  of  Marketing  and  Community 
Affairs 

2.  Acting  Director  of  CHSD  and  President  of  the  New  Center  Medical  Plaza 
Group 

3.  Audit  Review  Coordinator 

4.  Administrative  Assistant 

5.  Director  of  Research  and  Development 

6.  Director  of  Medical  Records  Department 
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The  interviews  were  not  audio  recorded  and  site  investigators  spent 
approximately  seven  hours  in  discussion  with  CHSD  personnel. 

The  following  topics  were  covered  during  the  interview: 

1.  Development  and  goals  of  the  QAP 

2.  Interplay  between  CHSD's  Physician  Service  Group  and  CHSD 
Administration  in  the  development  of  the  QAP 

3.  Staff  functions  and  responsibilities 

4.  Michigan  Department  of  Public  Health  audit 

5.  Funding  for  CHSD  and  QAP 

6.  Operational  level  of  the  QAP 

7.  Review  procedures  for  quality  assurance 

8.  Organization  of  QAP 

9.  Unique  characteristics  of  CHSD 
10.    Future  plans  for  the  QAP 

Quality  assurance  policies  are  developed  by  the  Acting  Medical  Director 
and  the  physicians  on  the  medical  Audit  Committee.    The  Acting  Medical  Director 
and  the  Executive  Vice  President  provided  most  information  on  the  present  sta- 
tus (developmental  and  operational)  and  future  plans  for  quality  assurance, 
whereas  the  other  interviews  provided  information  pertaining  to  actual  quality 
assurance  and  related  activities. 
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MARTIN  LUTHER  KING  HEALTH  CENTER 


Bronx,  New  York 


SECTION  I 
INTRODUCTION 


The  Dr.  Martin  Luther  King,  Jr.,  Health  Center  (MLK)  is  a  large  multi- 
specialty  neighborhood  health  center  originally  funded  by  the  Division  of 
Health  Affairs,  Office  of  Economic  Opportunity.    At  present,  MLK  is  advised 
by  an  incorporated  nonprofit  Community  Advisory  Board  and  controlled  by 
Montefiore  Hospital.    The  affiliation  with  Montefiore  was  stipulated  in  the 
original  0E0  grant  and  still  exists.    MLK  is  currently  funded,  in  part,  by 
a  large  Bureau  of  Community  Health  Services  grant. 

MLK  has  been  serving  its  target  population  since  August,  1966,  offering 
services  in  14  specialty  areas,  although  the  primary  goal  is  to  provide 
family-oriented,  team-centered  comprehensive  health  services  of  high  quality 
to  residents  of  South  Bronx,  New  York.    Specific  geographic  areas  are  desig- 
nated to  be  served  by  MLK  health  care  teams.    MLK  is  administered  by  a  project 
director,  who  receives  advisory  and  policy  input  from  the  Community  Advisory 
Board  and  Montefiore  Hospital  and  Medical  Center.    The  immediate  organiza- 
tional scheme  under  the  project  director's  office  is:    (1)  the  director  of 
health  services,  (2)  the  administrator  and  (3)  the  director  of  training. 
The  Health  Care  Evaluation  Committee  (HCEC)  in  the  department  of  Health 
Services  has  operational  responsibility  for  quality  assurance  activities  and 
held  its  first  meeting  February  15,  1971,  after  an  extensive  development 
period. 

Quality  assurance  at  MLK  centers  around  determination  of  patient  satis- 
faction, unusual  episode  review,  process  audit  of  charts,  and  outcome  audit. 
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SECTION  II 


QUALITY       ASSURANCE  PROGRAM 


The  discussion  of  MLK's  quality  assurance  program  is  divided  into  two 
major  sections: 

1 .  Background 

2.  Quality  assurance  program  components  formally  described  in 
documentation  supplied  to  study  staff  and  in  operation  as  of 
February  15,  1976 

Information  sources  for  No.  1  above  are  MLK  documents;  however,  some 
interview  data  are  used  to  add  information  dealing  with  other  planned  acti- 
vities.   This  information  is  added  only  to  documented  areas,  not  to  newly 
planned  areas  of  activity.    Information  sources  for  No.  2  are  interview 
data,  documents,  and  review  of  the  Health  Care  Evaluation  Committee  minutes. 


BACKGROUND 

The  Health  Care  Evaluation  Committee  (HCEC)  has  operational  responsibi- 
lity for  quality  assurance  activities,  as  well  as  decision-making  regarding 
review  results  conducted  jointly  with  the  director  of  health  services.  This 
committee,  originally  entitled  the  Medical  Care  Evaluation  Committee,  was 
established  in  February,  1971.    The  goals  were  to  evaluate  and  monitor  the 
quality  of  family  care  and  to  make  recommendations  to  practitioners  for 
improving  quality  of  care. 

The  history  of  the  committee  began  with  meetings  of  an  ad  hoc  group 
of  28  staff  personnel  concerned  enough  with  the  quality  of  care  to  discuss 
prospective  methods  and  operations  of  a  quality  assurance  program.  Because 
of  the  heterogeneity  of  this  group  and  its  lack  of  specific  direction,  formal 
action  was  not  taken  on  most  issues  raised.    An  MLK  researcher  who  was  a 
principal  force  in  this  original  committee  made  certain  recommendations 
concerning  the  importance  of  health  care  evaluations,  good  record-keeping 
in  charts  and  acceptable  practice  norms  to  the  Community  Health  Advocacy 
Department  (CHAD).    CHAD  had  the  responsibility  to  take  action,  in  conjunc- 
tion with  the  ad  hoc  group,  on  recommendations  made  for  quality  assurance 
methods  and  activities.    After  a  period  of  interaction,  checklist  audit  forms 
were  devised  based  on  recommendations  from  a  number  of  ad  hoc  committee  mem- 
bers.   The  format  of  these  forms  and  the  methodology  that  was  concurrently 
developed  eventually  led  to  the  establishment  of  the  present  audit  processes. 

At  this  juncture  in  the  development  of  quality  assurance  activities, 
Dr.  Lawrence  Weed,  the  developer  of  the  Problem  Oriented  Medical  Record  (POMR), 
visited  the  center  and  spoke  to  various  departments  about  the  use  and  imple- 
mentation of  his  system.    After  a  series  of  discussions  between  practitioners 
and  administrators,  a  modified  POMR  with  a  problem  list  for  each  chart  was 
adopted.    Implementation  of  the  POMR  is  important  in  the  development  of 
quality  assurance  activities  because  the  record  must  be  audited  to  insure 
proper  implementation  of  the  system. 
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After  POMR  implementation,  an  audit  sheet  was  designed  as  an  objective 
appraisal  of  family  as  opposed  to  individual  medical  records  and  was  used 
as  the  sole  tally  sheet  for  the  audit  procedures.    This  audit  process 
failed  to  specify  several  important  variables  including  the  size  of  charts, 
number  of  individuals  in  the  family  folder,  and  time  frame  of  care  or 
interval  between  visits.    Committee  members  reviewed  charts,  wrote  in  their 
findings  and  reported  this  information  in  weekly  committee  meetings.  In 
using  the  procedure,  it  became  apparent  that  approximately  42  charts  could 
be  reviewed  per  year,  and  that  some  teams  would  be  audited  more  frequently 
than  others.    With  these  problems  in  mind,  the  HCEC  undertook  a  reorgani- 
zation of  the  review  procedures. 

This  reorganization  was  to  focus  on  two  major  changes: 

1.  Development  of  a  new  audit  form 

2.  Change  in  review  procedures:    health  care  teams  would  begin 
reviewing  each  other's  performance  through  the  medical  record 
audit 

HCEC  hoped  these  revisions  would  lead  to  an  increased  number  of 
medical  record  reviews,  a  more  effective  educational  experience  for  the 
health  teams,  and  an  increase  in  self-awareness  on  the  part  of  the  teams. 

One  HCEC  member  undertook  the  responsibility  to  draft  new  protocols 
and  provide  a  more  complete  list  of  audit  components  for  the  chart  reviews. 
Documentation  noted  that  once  the  new  form  was  developed  by  this  member, 
the  whole  committee  met  to  approve  the  form.    After  discussion,  some  minor 
changes  were  made  and  the  committee  voted  to  adopt  the  new  form. 

In  terms  of  the  major  revision,  the  committee  decided  one  health  team 
should  review  another  team's  performance.    For  example,  the  Family  Health 
Worker  (FHW)  in  Team  A  might  review  the  FHW  notes  in  charts  from  Team  B  and 
internists  from  Team  A  might  review  internist  charts  from  Team  B.  Reviewers 
and  those  reviewed  are  required  to  sign  the  review  report,  allowing  the 
HCEC  to  be  sure  the  parties  have  met  and  discussed  the  review.    To  be  selected 
for  review,  a  family  record  must  have  at  least  six  individual  charts  in  the 
folder  and  there  must  be  at  least  one  adult  and  one  pediatric  chart.  Indi- 
viduals have  to  have  been  seen  within  the  previous  12-month  period.  The 
secretary  of  the  HCEC  is  responsible  for  assigning  the  random  charts.  The 
senior  FHW  is  responsible  for:    (1)  getting  the  charts  pulled,  (2)  assuring 
the  review  within  the  team  is  completed,  and  (3)  reporting  the  names  of  pro- 
crastinating reviewers  to  the  HCEC. 

At  present,  the  HCEC  is  composed  of  eight  full  members,  two  exofficio 
members  and  one  member  involved  as  needed.    These  positions  are  currently 
filled  by  the  following  personnel: 

1.  The  Chief  of  Internal  Medicine 

2.  The  Chief  of  Pediatrics 

3.  The  Chief  of  Dentistry 

4.  The  Chief  of  Nurse  Practitioners 
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5.  The  Chief  of  Family  Health  Workers 

6.  The  Chief  Pharmacist 

7.  The  Patient  Rights  Advocate 

8.  The  Secretary 

9.  The  Director  of  Health  Services  (exofficio  member) 

10.  Assistant  Director  of  Health  Services  (exofficio  member) 

11.  The  Chief  of  Obstetrics  and  Gynecology 

Plans  were  originally  made  to  include  a  representative  of  the  Community 
Advisory  Board  in  HCEC  activities,  but  this  was  impossible  to  arrange  since 
HCEC  meets  during  working  hours. 

Strict  confidentiality  concerning  HCEC  meetings  and  actions  is. enforced, 
and  it  is  made  clear  to  members  that  any  breach  of  this  confidentiality  could 
serve  as  a  basis  for  dismissal  from  the  committee. 


Formally  Planned  and  Implemented  Activities 

This  section  is  taken  largely  from  the  introductory  chapter  of  a  Heal th 
Care  Evaluation  Manual,  to  be  published  by  the  MLK  Health  Center  shortly. 
The  documentation  indicated  HCEC  meets  once  a  week  for  two  hours  to  evaluate 
medical  service  by  means  of  input  from  four  sources: 

1.  Determination  of  patient  satisfaction 

2.  Unusual  episode  review 

a.  Deaths 

b.  Restrictions  of  antibiotic  usage 

c.  Patient  complaints 

3.  Process  Audit  through  Chart  Review 

a.  Chart  standardization 

b.  Single  disease  entity  review 

c.  Comprehensive  family  chart  review 

4.  Outcome  audit 

The  documentation  notes  the  specific  outline  for  each  area  and  its 
designed  use. 


Determination  of  Patient  Satisfaction 

MLK  has  developed  a  simple  questionnaire  to  periodically  sample  (ran- 
domly) the  total  patient  population  to  determine  patient  satisfaction  with 
practitioners  and  other  MLK  personnel.    Response  rate  to  these  question- 
naires has  been  poor  and  has  given  MLK  only  a  limited  amount  of  information. 

Unusual  Episode  Reivew 

This  type  of  review  is  designed  to  be  a  systematic  check  of  undesirable 
outcomes  or  unusual  usage  of  certain  medications. 

Deaths.  All  deaths  that  come  to  the  attention  of  any  team  member 
must  be  reported  on  a  special  form  to  the  HCEC.    In  addition,  the 
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New  York  City  Medical  Examiner's  Office  sends  the  HCEC  a  list  of 
autopsied  persons  who  were  MLK  members  at  the  time  of  death. 
Regardless  of  method  of  notification,  it  is  mandatory  the  team 
obtain  as  much  information  as  possible  concerning  the  death  (e.g., 
narrative  details  from  observers,  a  medical  examiner's  report,  a 
hospital  summary,  a  hospital  autopsy  report).    This  procedure, 
coupled  with  chart  review,  is  used  to  establish  whether  any  over- 
sight or  lack  of  follow-up  may  have  led  to  the  death. 

The  same  form  may  be  used  to  report  a  serious  drug  reaction. 
This  is  designed  to  determine  if  medical  practice  changes  are 
necessary  to  prevent  recurrences. 

Restriction  of  Antibiotic  Usage.    MLK  felt  that  use  of  certain 
antibiotics  should  be  restricted  because  of  potential  toxicity, 
high  cost  of  the  drug  or  limited  therapeutic  indication.  Also 
present  was  the  concern  of  increasing  resistance  to  antibiotics 
in  certain  organisms.    A  form  to  collect  data  to  restrict  the 
use  of  cephalosporins,  clindamycin,  and  trimethoprim/sulfametho- 
xazole was  developed  by  the  chief  pharmacist  and  was  approved 
by  the  physician  groups.    Monitoring  is  performed  by  the  Formu- 
lary Committee  and  unusual  usage  of  these  medications  is  reported 
to  the  Health  Care  Evaluation  Committee.    If  the  reasons  for 
usage  are  considered  insufficient,  the  chief  of  service  resolves 
the  issue  with  the  practitioner  involved. 

Patient  Complaints.    Patient  complaints  often  lead  to  the 
uncovering  of  poor  technical  care  or  insensi tivi ty  in  delivery 
of  care.    The  documentation  indicates  a  full-time  patient 
advocate  is  employed  to  hear  patient  complaints  and  discuss 
them  at  HCEC  meetings.    Patients  are  informed  of  this  complaint 
procedure  through  a  booklet,  "Your  Rights  as  a  Patient,"  which 
they  receive  at  registration.    Such  investigations  can  be 
highly  controversial,  but  MLK  personnel  note  most  issues 
are  resolved  and  lead  to  better  communication  between  patient 
and  practitioner.    In  cases  of  gross  insensi tivity  on  the 
part  of  a  physician,  dismissal  can  occur. 

The  documentation  notes  several  problems  when  review  only 
examines  unusual  incidents  as  a  technique  for  maintaining  quality 
of  care.    First,  only  the  most  severe  errors  are  usually  found. 
Second,  a  physician  may  feel  he  is  being  selected  unfairly  by 
the  HCEC  since  the  investigation  of  unusual  incidents  is  not  a 
random  process. 

Process  Audit.    The  purpose  of  this  type  of  audit  is  to  examine 
charts  "as  a  reflection  of  treatment  patterns  both  for  specific 
diseases  and  for  total  patient  care."    The  documentation  notes 
three  areas  of  activity  under  process  audit:    chart  standardi- 
zation, single  disease  review,  and  comprehensive  family  chart 
review. 
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Chart  Standardization.    Several  years  ago,  the  problem-oriented 
medical  record  (POMR)  was  instituted  at  MLK.    Problem  sheets, 
flow  sheets,  and  Subjective  Objective  Assessment  Plan  (SOAP)  notes 
were  required.    To  insure  correct  usage  of  the  POMR,  audits  were 
performed  by  medical  assistants  on  charts  selected  in  random 
fashion. 

Three  simple  questions  were  asked: 

1.  Are  all  problems  including  health  maintenance  dated? 

2.  Does  the  most  recent  progress  noted  (by  practitioner 
being  reviewed)  have  each  problem  preceded  by  a  title? 

3.  Does  that  progress  note  have  all  problems  written  in 
SOAP  format? 

In  terms  of  the  specific  methodology,  a  scoring  system  has  been 
devised  to  produce  an  objective  measure  of  the  practitioner's 
performance  and  adequate  use  of  the  POMR.    This  clerical  audit 
involves  the  following  steps  and  score  sheets  (outlined  below): 

1.  The  medical  assistant  reviews  patient  charts  that  contain 
a  progress  note  and  the  problem  list  and  answers  the  three 
questions  on  the  POMR  Evaluation  Score  Sheet  (Figure  1). 

2.  The  audit  is  then  given  to  the  unit  manager  for  scoring 
(Figure  1 ) . 

3.  Unit  managers  keep  all  scored  audit  sheets  and  are  responsible 
for  insuring  that  all  medical  assistants  do  five  audits  per 
week  and  that  each  physician  and  nurse  has  between  three  and 
four  charts  audited  per  week  (Figure  2). 

4.  The  average  score  per  chart  for  each  practitioner  (MD  or 
RN)  is  calculated  by  the  unit  manager  on  a  quarterly  basis. 
Each  practitioner  is  informed  of  the  average  score.  If 
anyone's  average  is  less  than  4.0  he  or  she  is  offered 
additional  training  in  the  POMR.    If  practitioners  wish, 
they  can  be  referred  to  the  senior  family  health  worker  on 
the  team  or  her  representative  for  such  training. 

The  only  incentive  for  provider  change  was  competitive  peer 
pressure  to  achieve  higher  scores.    No  objective  assessment 
has  yet  been  made  to  assess  the  value  of  this  type  of  chart 
audit  for  producing  more  problem  oriented  charts;  however, 
MLK  staff  feel  there  are  fewer  charts  with  inadequate  problem 
lists  and  nonprobl em-oriented  notes. 

Single  Disease  Review.    Another  level  of  review  noted  in 
documents  deals  with  particular  disease  conditions.    This  review 
assumes  the  medical  records  are  accurate,  and  thus  auditable, 
since  accurate  charts  should  be  a  result  of  the  chart  standardi- 
zation audit  process. 

This  review  activity  requires  the  HCEC  to  select  a  disease 
entity  (no  formal  selection  process),  to  establish  criteria  for 
treatment  or  management  and  to  audit  the  performance  of  providers 
by  comparing  medical  record  information  with  the  criteria. 
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FIGURE  1 
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The  only  single  disease  review  occurred  in  1972  and  1973, 
when  a  Venereal  Disease  Committee  was  established  to  review  the 
diagnosis  and  treatment  of  gonorrhea.    The  committee  researched 
and  developed  a  specific  protocol  for  diagnosis  and  treatment, 
then  audited  a  number  of  medical  records  to  determine  the  per- 
formance levels  of  providers.    Performance  levels  were  expressed 
as  scores  given  to  providers  based  on  satisfying  specific  protocol 
requirements.    Scores  were  expressed  as  the  frequency  of  use  of 
specific  protocol  items,  reflecting  adequacy  of  diagnosis  and 
treatment. 

MLK  states  that  single  disease  review  as  an  approach  to 
assuring  quality  of  care  is  somewhat  limited.    In  general,  one 
concentrates  on  the  technical  aspects  of  treatment  of  the  disease. 
Further,  evaluation  of  the  practitioner's  sensitivity  as  to  what 
such  a  disease  means  to  the  patient  or  to  his  social  setting 
is  usually  impossible  to  analyze  from  a  single  encounter  in  the 
chart. 

Comprehensive  Family  Chart  Review.    The  basis  of  this  type  of 
process  audit  is  the  periodic  review  of  family  medical  records 
chosen  at  random.    The  selection  process  is  done  by  a  secretary 
of  HCEC.    The  chart  review  is  performed  by  one  team  (internist, 
pediatrician,  nurse  practitioner,  family  health  worker)  or 
another  on  a  monthly  basis.    A  standardized  audit  form  and 
instructions  have  been  designed  to  monitor  several  items: 

1.  Use  of  POMR  and  inclusion  on  problem  list  of  all  problems 

2.  Adequacy  of  health  maintenance  procedures 

3.  Adequacy  of  data  base,  assessment,  treatment  and  follow-up 
for  each  health  "problem" 

The  documentation  notes  that  for  such  audit  work,  certain 
standards  for  most  problems  have  been  worked  out  and  approved  by 
the  appropriate  professional  group.    These  standards  are  used  to 
provide  a  sense  of  peer  expectation,  which  is  the  basis  of 
discussion  of  differences  between  reviewer  and  reviewed.  For 
example,  minimal  health  maintenance  schedules  must  be  approved 
by  pediatric  and  internist  groups,  so  that  items  can  merely  be 
checked  off  as  done  or  not  and  properly  dated. 

1 .  Family  Audit  Form 

2.  Internal  Medicine  Review  Form 

3.  Pediatric  Review  Form 

The  Family  Audit  Form  is  a  four-page  summary  of  the  findings 
gathered  from  all  charts  of  the  entire  family.    The  Internal 
Medicine  and  Pediatric  Review  forms  are  one-page  worksheets  to 
be  used  for  individual  charts. 

The  Internal  Medicine  and  Pediatric  worksheets  are  divided 
into  four  sections:    Patient  and  Reviewer  Identification,  Health 
Maintenance,  Disease  Management  and  an  Overall  Assessment. 
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Patient  and  Reviewer  Identification.    This  section  is  self  explanatory. 


Health  Maintenance.    The  health  maintenance  section  is  a  tally  sheet 
for  indicating  whether  specific  requirements  of  health  maintenance 
protocols  have  been  followed.    It  can  be  reviewed  by  any  team  member. 
Items  which  have  been  done  can  be  so  indicated  by  checking  the  "yes" 
box  and  placing  in  the  box  the  most  recent  date  when  the  test  or  pro- 
cedure was  performed.    Space  is  available  under  "comments"  for 
clarification. 

Disease  Management.    This  part  of  the  form  is  completed  by  a  physician 
or  a  nurse  practitioner  and  physician.    The  reviewer  placed  in  the  boxes 
labeled  "Health  Problems"  the  items  written  on  the  problem  sheet  of  the 
chart.    For  each  of  these  problems  he  can  then  indicate  by  checking 
the  YES  column  that  the  problem  is  on  the  problem  list,  and  in  the  YES 
or  NO  column  whether  there  is  an  adequate  data  base,  assessment,  and 
follow-up  treatment.    If  any  of  these  areas  are  inadequate,  the 
reviewer  must  check  NO  and  document  the  opinion  by  commenting  in  the 
adjacent  free  space.    In  reviewing  a  chart,  a  physician  may  find  that 
several  problems  have  not  been  placed  on  the  problem  list.    In  this 
case,  the  unrecorded  problem  is  written  in  the  health  problem  box 
and  NO  is  checked  after  it. 

Overall  Assessment.    The  physician  uses  this  space  for  his  general 
assessment.    Here  recommendations  for  improvement  are  made  to  the 
reviewed  physician  and  entire  health  team.    Equally  important  comments 
of  praise  for  good  care  can  be  made  here  as  well. 

Comments  should  include: 

1.  Adequacy  of  health  maintenance  to  date,  tests  needed  to  be  done 

2.  Use  of  problem-oriented  notes  and  problem  sheet,  specific  comments 

3.  Disagreements  concerning  diagnosis,  follow-up  and  treatment 

Comments  in  the  "Overall  Assessment"  may  be  typed  verbatim  onto 
the  Family  Audit  Form  under  the  section  "Review  Summary  of  Individual 
Family  Members."    In  discussing  the  chart  review  with  the  reviewed 
team,  the  reviewing  team  uses  these  worksheets  to  document  findings. 

After  the  formal  review  of  a  family  chart,  the  reviewing  team  must 
discuss  findings  with  the  reviewed  team.    It  is  at  this  point  that  dif- 
ferences of  opinion  are  discussed  and  documented.    After  this  inter- 
action, the  health  team  review  results  are  sent  to  the  HCEC  for  final 
review.    If  the  chiefs  of  service  find  any  inadequacy  in  the  review,  they 
then  discuss  the  review  with  reviewer,  reviewed  or  both.    Final  follow- 
up  of  reviewed  problems  is  referred  to  the  director  of  health  services. 

Family  chart  audits  have  been  performed  at  MLK  almost  monthly. 
This  allows  review  of  approximately  12  charts  per  team  (there  are  eight 
teams)  per  year  and  is  used  as  an  educational  tool  for  team  members. 
Such  an  audit  occasionally  identifies  a  practitioner  who  consistently 
performs  poorly.    Such  a  person  may  have  an  intensive  review  of  charts 
performed  by  the  chief  of  his  service  and  be  given  a  warning  notice  of 
termination  or  suggestions  for  improvement  and  a  date  for  reevaluation. 
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Some  problems  with  a  comprehensive  family  chart  review  have  been 
noted  by  MLK  personnel.    First  it  requires  extra  effort  on  the  part  of 
teams  to  perform  the  review  itself  and  then  to  find  time  to  discuss  the 
review  with  tightly  scheduled  members  of  the  reviewed  team.    It  takes 
weekly  meetings  of  the  HCEC  and  constant  checking  of  chart  reviews  by 
the  chiefs  of  service.    A  significant  factor  is  the  small  number  of 
charts  actually  being  reviewed  through  this  process  (88  charts  were 
reviewed  during  the  past  year). 

Outcome  Review.    The  single  disease  entity  whose  outcome  is  presently 
under  review  is  hypertension.    This  study,  partially  operational  now,  is 
designed  to  accomplish  the  following  goals: 

1.  Screen  adult  charts  at  MLK  to  see  what  proportion  of  hypertensive 
patients  are  under  adequate  control 

2.  Develop  a  manual  system  for  more  careful  follow-up  and  surveillance 
of  all  hypertensives  at  MLK 

3.  Prove  that  such  a  system  will  improve  the  percentage  of  hypertensive 
patients  with  controlled  blood  pressure  at  MLK 

4.  Develop  a  community  participatory  patient  education  project 

5.  Prove  that  such  an  educational  project  will  further  improve  control 
of  blood  pressure  in  hypertensive  patients  beyond  that  achieved 

by  the  surveillance  system 

6.  Generate  more  patient  visits  to  MLK  by  assuring  hypertensive 
patients  are  more  carefully  followed.    Better  utilize  nurse 
practitioners  in  following  hypertensives. 

Periodically  a  list  of  physicians,  nurses,  and  family  health  workers 
will  be  published  with  the  percentage  of  their  hypertensive  patients 
whose  blood  pressure  is  under  control.    It  is  hoped  that  the  peer  compe- 
tition will  work  to  increase  the  fraction  of  hypertensives  with  controlled 
blood  pressure. 

MLK  personnel  identified  several  problem  areas  in  all  quality 
assurance  activities,  including: 

1.  The  difficulty  of  keeping  health  teams  interested  in  conducting 
rev  i  ews 

2.  Assuring  review  teams  the  audit  is  a  worthwhile  endeavor 

3.  Selecting  the  type  of  sanction  to  use  for  noncompliance 

4.  Gaining  coooperation  among  providers  in  identifying  specific 
problems  for  study 


ADDITIONAL  COMMENTS 

Quality  assurance  activities  at  MLK  are  at  many  stages  of  implementation. 
The  documentation  of  operational  areas  was  limited,  and  site  investigators 
depended  on  separate  interviews  to  establish  operational  status.  Generally, 
activities  noted  in  documents  and  interview  data  were  being  implemented. 
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SECTION  III 


DELIVERY       SYSTEM  CHARACTERISTICS 


The  Martin  Luther  King  Health  Center  delivers  a  comprehensive  set  of  medi- 
cal services  using  the  internist  and  pediatrician  as  principal  primary  care 
physicians  on  health  teams.    A  typical  health  care  team  includes: 


Public  Health  Nurses 
Pediatricians 
Internists 
Secretary 
Dentist 


A  typical  unit  serving  two  teams  includes: 


There  are  eight  teams  at  MLK  each  serving  a  population  of  about  11,700 
registered  families. 

In  addition  to  the  primary  care  physicians  working  within  a  health  team, 
12  specialties  are  offered  at  MLK.    They  are: 


1. 

Dermatology 

2. 

Ear,  Nose  and  Throat 

3. 

Allergy 

4. 

Podiatry 

5. 

Obstetrics  and  Gynecology 

6. 

Electrocardiography 

7. 

Orthopedics 

8. 

Ophthalmology 

9. 

Psychiatry 

10. 

Surgery 

n. 

Speech  and  Hearing 

12. 

Dentistry 

The  patient  population  served  by  MLK  is  primarily  of  lower  economic 
background.    Sixty  percent  of  the  population  is  black  and  approximately 
forty  percent  is  Puerto  Rican.    Based  on  the  most  recent  comprehensive  stat- 
istics available  (1973)  from  MLK,  the  population  may  be  broken  down  as 
follows: 

Patient  Information    Residents  in  area  served  90,000 


Family  Health  Workers 


4 


Medical  Assistants 
Receptionists 


7 
3 


Individuals  in  households 
registered  and  receiving 
care  in  past  year 


29,000 
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Maximum  distance  to  main 
health  center  by  blocks 


13 


Percentage  of  patients 
living  within  five  blocks 
of  main  health  center 


80 


Patients  registered  per  team 


3,600  -  5,800 


Patient  Services 


Visits  to  Health  Center 
Health  care  units 
Emergency  room  screening 
Specialty  Unit 
Dental  Unit 

Home  visits  by  family  health  worker 
Pharmacy  prescriptions 
Lab  exams 
X-Rays 

Transportation  for  disabled 


182,000 
84,000 
49,000 
27,000 
22,000 
40,000 

221 ,000 
96,000 
20,000 
12,000 


Staff  (FTE*) 


Total  number  of  employees 

Health  Services 

Administration 

Community  Health  Advisory 

Training 


432 
329 
90 


7 
6 


A  large  proportion  of  the  staff  is  black  and  Puerto  Rican  (90%)  and 
approximately  half  of  the  patient  contact  staff  speak  Spanish. 


REVENUE  SOURCES 

MLK  is  financed  primarily  by  a  Community  Health  Services  grant  of  approxi- 
mately $3,000,000  and  through  Medicaid  capitation  reimbursements  of  about 
$5,000,000.    Additional  grants  and  revenue  sources  amount  to  approximately 
$500,000.    These  monies  are  derived  primarily  from  Medicare,  a  grant  from  the 
Robert  Wood  Johnson  Foundation  and  the  Department  of  Commerce.    Total  money 
allocated  for  HCEC  work  (based  on  percentage  of  time  spent  on  HCEC  activities) 
is  approximately  $24,000. 


ORGANIZATIONAL  STRUCTURE 

The  office  of  director  of  health  services  comes  directly  under  the  project 
director  who  supervises  the  assistant  director  of  health  services  for  pro- 
fessional affairs  and  the  supportive  services  offered  at  MLK. 

Direct  services  are  administered  by  the  health  care  teams  and  the  spec- 
ialty units.    Supportive  services  include  X-ray,  laboratory,  pharmacy,  medical 
records  and  registration.    In  addition,  the  director  of  health  services  super- 
vises the  office  of  professional  affairs. 


Full-time  equivalent 
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The  administrator  has  responsibility  for  operations  analysis,  business 
services  (accounting,  payroll,  purchasing,  etc.)  and  central  services 
(engineering,  housekeeping,  etc.). 

The  director  of  training  also  reports  directly  to  the  department  project 
director. 
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SECTION  IV 

HCMS       STUDY  INFORMATION 


MLK  was  initially  contacted  by  mail  November  21,  1975,  and  agreed  to 
participate  in  a  letter  dated  December  3,  1975.    Confirmation  of  MLK's 
participation  was  obtained  in  a  telephone  conversation  on  December  5,  1975. 
Initial  documentation  of  the  MLK  program  was  received  in  the  form  of  the 
three  most  recent  annual  reports. 

The  documentation  received  and  compiled  included: 


1.  The  4th,  5th  and  6th  Annual  Reports,  1970  to  1972 

2.  "Assuring  Quality  in  Ambulatory  Care,"  an  introductory  chapter 
for  the  Health  Care  Evaluation  Manual  to  be  published  soon 

3.  A  statement  concerning  the  history  and  development  of  the  HCEC 

4.  Report  of  the  HCEC  1973  to  1974 

5.  Minutes  of  the  HCEC,  November  25,  1974  and  March  12,  1975 

6.  Cost  breakdown  for  HCEC  committee  work 

7.  Comprehensive  Family  Audit  Instructions 

8.  Description  of  the  Problem-Oriented  Medical  System  used  at  MLK 

9.  Problem-Oriented  Medical  Record  Checklist 

10.  Instructions  for  the  clerical  audit  procedures 

11.  Review  form  checklist  for  auditing  (a)  prenatal  care,  (b)  gynecologic 
problems,  (c)  postpartum  visits  and  (d)  family  planning  visits 

12.  Information  on  flow  charts  for  an  ongoing  hypertension  study 


These  documents  were  used  as  collaborative  evidence  in  constructing  the 
description  of  MLK's  program.    The  major  information  obtained  on  MLK's  pro- 
gram was  collected  during  a  series  of  interviews  with  the  personnel  directly 
responsible  for  quality  assurance  activities,  as  well  as  with  personnel 
responsible  for  total  program  administration.    Interviews  were  conducted 
with  the  following  MLK  personnel: 

1.  Director  of  Health  Services 

2.  Assistant  Director  of  Health  Services  for  Professional  Affairs 

3.  Chief  of  Pediatrics  and  Chairman  of  the  HCEC 

4.  Chief  of  Internal  Medicine  and  member  of  the  HCEC 

5.  Senior  Family  Health  Worker 


Topics  covered  during  the  interviews  included: 


1.  The  history  of  quality  assessment  at  MLK 

2.  The  function  and  operation  of  the  health  team 

3.  Social  and  demographic  factors  which  influence  MLK's  operation 

4.  The  use  of  the  POMR 

5.  Current  status  of  quality  assessment  at  MLK 

6.  Case  selection  for  review 

7.  Feedback  to  practitioners 

8.  Cost  of  operating  a  quality  assessment  activity 

9.  Special  studies 


200 


HARRIS  COUNTY  NEIGHBORHOOD  HEALTH  CLINICS 
Houston,  Texas 


SECTION  I 
INTRODUCTION 


The  Harris  County  Hospital  District  operates  seven  neighborhood  health  clinics 
in  Houston,  Texas.    The  clinics  are  supported  by  the  county  tax  base,  as  well  as 
various  Department  of  Health,  Education  and  Welfare  (DHEW)  grants  and  third  party 
payers.    The  district,  while  supplying  fiscal  support,  depends  on  the  Department 
of  Community  Medicine  (DCM),  Baylor  College  of  Medicine,  to  provide  medical  re- 
sources within  the  clinics.    All  physicians  working  in  the  clinics  are  employed 
through  the  Department  of  Community  Medicine  and  report  to  its  chairman.  The 
administration  of  the  clinics  is  conducted  by  hospital  district  personnel,  who 
report  to  the  district  administrator. 

The  first  neighborhood  health  clinic  was  supported  in  part  by  a  Comprehen- 
sive Health  Planning  314(e)  demonstration  grant  from  the  DHEW  and  the  whole 
network  was  initially  aided  in  part  by  an  Office  of  Economic  Opportunity  (0E0) 
planning  grant.    These  grants  were  given  to  the  hospital  district,  which  in  turn 
joined  with  the  DCM  to  begin  delivering  services  to  medically  indigent  county 
residents. 

The  seven  clinics  are  affiliated  with  Ben  Taub  General  Hospital  and  Jefferson 
Davis  Hospital.    There  is  fairly  close  coordination  between  physicians,  through 
the  DCM,  in  delivering  services  to  all  seven  clinics. 

The  focus  of  the  study  is  on  the  DCM  Peer  Review  Committee,  which  is  respon- 
sible for  implementing  quality  assurance  methods  for  each  of  the  seven  clinics. 
Various  quality  assurance  activities  have  been  conducted  by  the  DCM  since  1970. 

The  focus  of  this  description  is  twofold:    (a)  to  describe  the  history  of 
developmental  activities,  and  (b)  to  describe  the  present  audit  system  (since 
January,  1976).    Audit  responsibility  resides  in  the  Peer  Review  Committee, 
which  includes  five  center  physicians  and  one  medical  records  person  from  Ben 
Taub  General  Hospital.    The  committee  was  formed  in  January,  1976,  after  the 
1975  committee  completed  its  audits. 

Site  investigators  visited  five  of  the  clinics:    (a)  Settegast  Clinic, 
(b)  Casa  de  Amigos,  (c)  Acres  Home,  (d)  Sunnyside  and  (e)  Ripley  House  Clinic. 
Physician  staff  members  were  interviewed  at  each  clinic  regarding  present  audit 
procedures. 

The  Harris  County  Neighborhood  Health  Clinics  (HCNHC)  were  included  in  the 
study  because  of  the  unique  implementation  of  quality  assurance  activities  through 
a  department  of  a  medical  school  and  the  long  historical  record  of  quality  assur- 
ance activities. 
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SECTION  II 


QUALITY     ASSURANCE  PROGRAM 


This  section  will  describe  two  aspects  of  the  HCNHC  peer  review  system: 
(a)  the  development  and  implementation  of  the  method  during  1972;  (b)  the  pre- 
sent operational  peer  review  system,  implemented  in  January,  1976. 

The  development  of  quality  assurance  activities  within  the  neighborhood 
health  centers  parallels  the  amount  of  collaboration  with  the  Department  of 
Community  Medicine.    In  1972,  the  chairman  wanted  to  upgrade  the  quality  of 
medical  records  by  establishing  a  routine  data  base  for  the  records,  with  a  new 
record  system  which  required  a  specific  format  for  a  corresponding  number  of 
specified  medical  data  elements. 

During  1971  and  1972  the  DCM  received  a  grant  from  the  National  Center  for 
Health  Services  Research  and  Development  (NCHSRD  -  DHEW)  to  develop  a  computerized 
patient  management  system  for  a  neighborhood  health  center.    Members  of  the  DCM, 
while  developing  the  health  information  system,  also  developed  a  peer  audit  pro- 
gram.   Initial  program  emphasis  was  on  a  systematic  data  base  for  the  audit  pro- 
gram.   The  official  program,  Health-Illness  Profile,  was  designed  to  consider 
all  elements  of  a  patient-physician  encounter.    All  encounter  data  are  key-punched 
at  the  clinics,  printed  at  DCM  (Baylor),  and  sent  back  to  the  clinics. 

Although  the  Health-Illness  Profile  is  not  presently  used  in  audit  activities, 
Problem  Oriented  Medical  Records  (POMR)  used  in  the  audits  are  an  outcome  of  the 
development  of  a  systematic  computerized  data  collection  and  reporting  system. 

In  1972,  the  DCM  formally  established  a  peer  review  system  in  one  neighbor- 
hood clinic,  Casa  de  Amigos.    The  methodology  and  purpose  of  the  peer  review 
system  was  described  in  "A  System  for  A  Neighborhood  Health  Center  Peer  Review" 
by  DCM  physicians. 

This  section  will  describe  the  initial  system  components  from  the  less  so- 
phisticated 1972  document,  as  well  as  the  1974  and  1975  audit  programs.    The  1972 
program,  implemented  at  Casa  de  Amigos,  was  designed  to  conduct  medical  record 
audits  for  completeness  and  correctness  (Phase  I),  to  conduct  peer  review  for 
disease  conditions  by  employing  explicit  standards  (Phase  II)  and  to  use  the 
Health-Illness  Profile  to  measure  performance  of  entire  health  teams  (Phase  III). 
Briefly,  the  documents  noted  that  the  three  phases  accomplish  the  following 
items: 

Phase  I.    From  a  base  population  of  approximately  2,000  patient  clinic 
records  stratified  by  age  and  sex,  a  random  sample  of  200  patient  records  was 
drawn.    These  records  were  reviewed  against  an  itemized  checklist  for  complete- 
ness and  correctness.    An  arbitrary  criterion  was  established  requiring  that  at 
least  90  percent  of  the  items  on  the  checklist  be  completed  after  four  clinic 
visits.    Reaching  this  level  of  acceptance  qualified  the  record  to  be  noted  as 
complete. 
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Phase  II.    This  phase  required  the  audit  of  medical  records  for  specific 
informational  elements  (data  base)  using  explicit  criteria.    The  audit  was  con- 
structed around  the  format  of  the  POMR  used  in  the  clinics.    These  elements  are 
well  known  and  include: 

1 .  Problem  1 ist 

2.  Data  base 

3.  Plan  for  care 

4.  Follow-up  plans  and  actions 

Criteria  were  then  developed  for  13  disease  entities  which  comprised  70%  of  the 
visits  for  Casa  de  Amigos'  adult  and  pediatric  population.    Standards  were  devel- 
oped by  revising  already  documented  standards,  which  were  then  revised  by  DMC  phy- 
sicians for  use  in  the  Neighborhood  Health  Center.    After  departmental  meetings, 
the  standards  were  presented  and  discussed  with  physicians  in  the  clinic.  In 
addition,  they  were  analyzed  and  discussed  individually  with  each  specialist 
for  modification.    After  a  three-month  test  period,  the  standards  were  implemented 
and  reviewed. 

The  second  and  most  important  feature  of  this  phase  was  professional  peer 
review  of  the  abstracted  information  comparing  standards  with  data  taken  from 
medical  records.    The  DCM  Peer  Review  Committee  reviewed  the  results  with  the 
goal  of  addressing  total  provider  performance  by  considering  the  following  ele- 
ments: 

1.  Completeness  of  data  base 

2.  Completeness  of  physiological,  psychological  and  social  problem  list 

3.  Appropriateness  of  diagnosis 

4.  Adequacy  of  treatment 

5.  Evidence  of  health  education 

6.  Redundancy  of  prescriptions  or  laboratory  tests 

7.  Continuity  of  care 

8.  Proper  referrals  and  consultations 

DCM  physicians  devised  a  rating  system  based  on  these  eight  items.  The 
rating  system  assigned  a  total  of  300  points  across  the  eight  operations,  with 
items  one  and  two  receiving  one-third  of  the  points,  items  three,  four  and  five 
receiving  one-third,  and  items  five,  six,  seven  and  eight  receiving  the  final 
one-third  of  the  possible  300  points.    Peer  review  physicians  assign  scores  to 
each  medical  record,  based  on  the  criteria  comparison.    This  attempt  to  assign 
quantitative  scores  to  performance  was  done  to  support  peer  review  judgments  made 
during  the  audit  procedures.    The  document  indicated  eight  stated  questions  were 
divided  among  the  four  sections  of  the  POMR  in  order  to  assist  record  keeping 
and  evaluating  specific  diagnoses  and  problems.    Figure  1  graphically  displays 
the  rating  scale.    For  each  of  the  13  problems  selected  for  review,  criteria 
were  developed  for  each  of  the  eight  questions. 

It  is  important  to  note  that  this  system  made  a  critical  distinction  between 
standards  and  specific  criteria.    Because  of  the  use  of  POMR,  four  basic  sections 
must  be  documented.    In  addition  to  these  sections,  and  depending  on  the  type  of 
patient  (e.g.,  adult,  pediatric)  and  problem  or  diagnosis,  there  were  more 
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FIGURE  1 


HCHD  NO. 


AGE 


SEX 


RATING  SCALE  FOR  A  PATIENT'S  PROBLEM 
OR  DIAGNOSIS  AND  CHART  EVALUATION* 


CHART 
EVALUATION 

QUESTION 
NUMBER 

ITEM 

RATING 

BASIC 

1 

Data  Base 
Minimum  Standard 

50 

INFORMATION 

2 

Problem  List 

50 

Subtotal 

100 

3 

Appropriateness 
of  Diagnosis 

40 

INITIAL 

4 

Adequacy  of 
Treatment 

40 

PLAN 

5 

Evidence  of 
Health  Education 

20 

Subtotal 

100 

6 

No  Redundancy 

20 

FOLLOW-UP 

7 

Continuity  of 
Care 

50 

8 

Referrals  and/or 
Consul tations 

30 

Subtotal 

100 

TOTAL 

8  quantities 

300 

*From  "A  System  for  a  Neighborhood  Health  Center  Peer  Review" 
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specific  standards  for  documenting  the  standard  in  the  POMR  for  specific  en- 
counters.   Each  POMR  standard  had  a  corresponding  criterion  which  indicated 
necessary  data  elements  for  that  specific  standard.    For  example,  when  a  review 
physician  assesses  a  POMR  for  the  Pediatric  Standard  Data  Base  (Figure  2),  the 
review  items  are  the  POMR  standards  which  must  be  present  in  the  form  of  the  cri- 
teria noted  in  the  corresponding  row.    The  criteria  are  much  more  specific  and 
provide  assistance  to  the  reviewing  physician  in  making  an  assessment  (rating) 
for  a  particular  POMR  or  episode  of  care. 

After  assessing  a  medical  record  and  assigning  scores  among  the  eight  ques- 
tions, a  range  for  the  scores  established  by  the  DCM  divided  medical  care  into 
three  categories: 

1.  Excellent  (score  270  to  300) 

2.  Adequate  (score  240  to  269) 

3.  Inadequate  (less  than  240) 

This  scaling  procedure  allowed  review  physicians  and  others  to  rank  medical 
records  into  easily  identifiable  categories. 

These  procedures  were  designed  not  only  to  evaluate  individual  charts,  but 
also  to  provide  a  means  for  reviewing  care  received  by  a  group  of  patients  with 
the  same  diagnosis  or  problem.    The  documents  note  that  the  peer  review  physi- 
cians could  review  medical  records  of  a  sample  of  diabetic  patients  in  order  to 
determine  the  number  of  records  meeting  both  POMR  standards  and  specific  criteria 
in  the  peer  review  process. 

The  criteria  for  categories  (e.g.,  excellent,  adequate,  inadequate)  changed 
when  a  group  of  charts  was  assessed.    Care  was  judged  to  be  excellent  if  90% 
or  more  of  the  charts  were  classified  as  excellent;  adequate  if  90%  or  more  of 
the  charts  were  classified  as  excellent  or  adequate;  and  inadequate  if  less  than 
90%  of  the  charts  were  classified  as  excellent  or  adequate.    This  group  rating 
scheme  is  based  on  the  300-point  scale  outlined  earlier.    If  a  number  of  indi- 
vidual records  within  the  group  (greater  than  90%)  ran  scores  between  270  and  300, 
then  the  group  would  be  rated  excellent.    The  same  point  intervals  for  adequate 
and  inadequate  were  used  in  determining  group  performance.    The  document  further 
stated  that  two  separate  Peer  Review  Committees  would  evaluate  the  same  group  of 
charts  and  that  the  results  obtained  from  both  committees  would  be  compared  and 
analyzed. 

The  document  outlined  an  educational  program  that  would  be  implemented  to 
correspond  with  the  peer  review  system.    Plans  were  made  to  institute  an  educa- 
tional program  at  Baylor  College  of  Medicine  to  educate  physicians  on  specific 
problem  areas  identified  during  the  reviews.    It  was  the  plan  of  the  program  to 
include  seminars,  in-service  training  at  Ben  Taub  General,  Jefferson  Davis  Hospital 
and  other  hospitals  in  the  Texas  Medical  Center,  as  well  as  discussions  and 
regular  scientific  meetings  for  neighborhood  health  center  physicians.    Also  in- 
cluded in  this  blueprint  were  plans  to  distribute  current  literature  on  pertinent 
matters  to  the  group.    The  purpose  of  the  educational  system  would  be  to  take 
advantage  of  audit  results  that  identify  problem  areas  within  the  care  system, 
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FIGURE  2 

SCALE  AND  CRITERIA  FOR  EVALUATING 
THE  PEDIATRIC  STANDARD  DATE  BASE* 


REVIEW 
ITEM 

RATING 

CRITERIA  TO  BE  USED 

HISTORICAL 

Chief  Complaint 
and 

Present  Illness 

8 

The  chief  complaint  might  be 
included  in  present  illness. 
Onset  of  present  illness  should 
be  stated.    Present  illness 
should  be  organized  chronologi- 
cally. 

Patient's 
History 

10 

Pregnancy,  birth,  neonatal 
feeding  and  nutrition  under  one 
year  of  age.    Growth  chart  and 
development.    Immunizations,  Ill- 
nesses, operations,  allergies. 

Family  History 

7 

Tuberculosis,  allergy  (asthma, 
hay  fever),  diabetes,  anemia 
(sickle  cell  included),  kidney 
disease  should  be  stated  if 
present  or  not  in  family  history. 

Subtotal 

25 

The  full  rating  will  be  given 
if  all  the  historical  aspects 
are  included. 

Anthropometry , 
Vital  Signs  and 
Appearance 

5 

Head  circumference  will  be 
included  in  anthropometric 
measures  in  children  under  one 
year.    Blood  pressure  should 
not  be  considered  (vital  signs) 
in  children  under  five  years. 

H.E.E.N.T.** 

5 

These  five  aspects  of  the 
physical  examination  should  be 
present. 

PHYSICAL 
EXAMINATION 

i 
I 

Chest:  Lungs, 
Heart 

5 

Breast  examination  should  be 
part  of  chest  examination  in 
female  teenagers. 

Abdomen , 
Genital ia, 
Rectal 

5 

The  abdominal  examination  will 
be  the  main  part  of  these  three 
components,  especially  after 
the  six-weeks'  physical  exami- 
nation.   Rectal,  if  specific 
indication 

1  

Extremities, 
Skin  and  Neurological 

0 

The  neurological  examination 
will  be  performed  if  the  child 
has  specific  indication  for  it. 

Subtotal 

25 

The  full  value  will  be  given  if 
the  physical  examination  was 
completed  in  accordance  with 
stated  criteria.  Abbreviations 
in  common  use  are  acceptable. 

i 

i 

j 

TOTAL 

50 

This  value  will  represent 
addition  of  historical  and 
physical  part  of  the  standard 
data  base. 

*From  "A  System  for  a  Neighborhood  Health  Center  Peer  Review" 
**Hearing,  eyes,  ears,  nose,  throat 
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and  ultimately  correct  those  problems  with  positive  educational  inputs  to  pro- 
viders.   It  was  further  stated  that  a  program  of  this  type  could  be  expanded  to 
all  neighborhood  clinics  of  the  Community  Medicine  Service  of  the  Harris  County 
Hospital  District. 

Phase  III.    The  final  phase  of  the  peer  review  program  evaluated  medical 
services  by  using  the  Health-Illness  Profile.    Phase  III  was  designed  to  analyze 
the  correlation  between  medical  record  standards  and  specific  criteria  elements 
against  medical  encounter  information  contained  in  this  computerized  profile. 
The  document  notes  the  major  objectives  of  the  Health-Illness  Profile  are  to: 
(a)  present  complete,  concise,  and  accurate  information  on  the  state  of  health 
and  illness  of  all  enrollees,  (b)  provide  a  basis  for  a  peer  review  program 
and  (c)  facilitate  the  compilation  of  statistical  reports  useful  to  the  overall 
clinic  management.    The  Health-Illness  Profile  contains  the  following  elements 
(Figure  3) . 


FIGURE  3 
HEALTH-ILLNESS  PROFILE  ELEMENTS 


1 .  Identification 

Identification  number,  name,  address,  date  of  birth,  race 

2.  Health  Profile 

a.  Physical  and  anthropometric  characteristics  (height,  weight) 

b.  Functional  capacity  of  specific  systems  (respiratory,  audio- 
visual, cardiovascular,  etc.) 

c.  Dental  condition 

d.  Fertility  history 

e.  Immunization  status  (vaccinations  and  illnesses  that  confer  per- 
manent immunity) 

f.  Mental  competency  or  developmental  status 

g.  Socioeconomic  situation  (family,  employment  and  education) 

3.  Illness  Profile 

a.  Active  problems  or  medical  diagnoses  with  date  of  onset 

b.  Inactive  problems  or  medical  diagnoses  with  date  of  termination 

c.  Medication  history 

4.  Services  Rendered 

a.  List  of  clinic  visits 

b.  Hospitalization  list 

c.  List  of  surgical  interventions 

5.  Community  Descriptive  Data 

Key  health  indices  of  the  patient's  census  tract 
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The  purpose  of  Phase  III  is  to  implement  components  of  the  Health-Illness 
Profile,  as  developed  for  general  medical  information  uses,  and  for  quality  as- 
surance purposes.    The  document  notes  that  the  Health-Illness  Profile  "will 
yield  immediate  and  automated  information  concerning  medical  care,"  as  well  as 
linking  this  review  with  specific  patient  episodes.    The  main  thrust  of  the  pro- 
file is  to  reduce  the  amount  of  time  spent  in  the  clerical  portion  of  peer  review 
activities. 

The  program  elements  outlined  above  as  Phase  I  and  II  were  partially  imple- 
mented in  1974,  1975  and  1976,  as  well  as  the  initial  implementation  in  1971-1972 
Phase  III  has  not  been  implemented.    The  audits  implemented  between  1974  and  1976 
varied  in  the  specific  components  used,  extensiveness  of  the  program,  subject  of 
audits,  actions  taken  based  on  results  and  DCM  personnel  involved.    The  descrip- 
tive focus  is  on  the  peer  review  system  implemented  on  January  1,  1976.  The 
present  audits  were  the  subject  of  most  on-site  data  collection. 


OPERATIONAL  ACTIVITIES 

The  organizational  format  for  implementing  audit  procedures  includes  one 
central  Peer  Review  Committee  (DCM  physicians)  which  has  responsibility  for  im- 
plementing and  assessing  audits  for  all  seven  neighborhood  health  centers.  The 
previous  audits  required  committee  members  to  collect  record  information  (ab- 
stracting), compile  the  data,  and  assess  the  information.    Although  the  1976 
audit  involved  one  central  committee,  it  included  seven  residents  who  were  charged 
with  the  responsibility  to  collect  and  compile  all  record  information.    The  com- 
mittee members,  especially  the  chairman,  supervised  the  residents  during  the 
data  collection,  but  did  not  have  to  collect  the  information. 

The  1976  audit  centered  around  the  review  of  seven  disease  conditions  (four 
chronic,  three  acute).    The  goal  for  the  1976  audit  was  broadened  somewhat  from 
previous  audits,  to  include  not  only  measurement  of  effective  use  of  POMR's,  but 
also  to  determine  whether  routine  ambulatory  problems  are  being  properly  managed. 

Explicit  criteria  were  used  to  measure  performance  and  as  a  guide  to  the 
record  abstractors.    The  present  chairman  of  the  committee  had  several  meetings 
with  the  immediate  past  chairman  to  review  and  revise  already  developed  criteria. 
It  was  noted  that  the  former  chairman  spent  much  time  in  preparation  for  the  1975 
audit  developing  criteria,  including  sending  each  HCNHC  physician  a  copy  of  the 
criteria  and  requesting  comments  for  revision.    After  the  two  chairmen  revised 
the  criteria,  they  were  presented  to  the  new  committee  for  discussion  and  further 
revisions. 

Once  the  criteria  (i.e.,  standards  for  management  of  the  seven  disease  con- 
ditions) were  agreed  upon,  three  members  of  the  Peer  Review  Committee  met  with 
the  seven  residents  to  explain  the  purpose  of  the  audit  and  the  data  collection- 
compilation  procedures.    After  lengthy  discussion,  the  residents  went  to  their 
respective  clinics  to  begin  audit  responsibilities  (as  well  as  provider  responsi- 
bilities).   The  purpose  of  having  residents  take  a  major  role  in  data  collection 
and  compilation  was  to  reduce  variability  in  implementing  the  audits.    HCNHC  per- 
sonnel indicated  this  had  been  a  problem  with  past  audits. 
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The  criteria  items  for  each  condition  had  a  priority  scheme  of  three  cate- 
gories: 

PI  Highest 

P2  Intermediate 

P3  Lowest 

For  each  criterion,  the  abstractor  examines  the  medical  record  and  notes  "pre- 
sent (+)"  or  "absent  (-)"  on  each.    The  scoring  for  each  medical  record  is  computed 
by  determining  the  percent  of  items  present. 

In  selecting  records  for  audit,  the  method  requires  that  100  records  be 
selected  for  each  condition,  that  a  random  sample  of  20  be  chosen  for  audit  for 
each  condition,  and  that  a  date  be  chosen  to  establish  a  time-frame  for  the  audits. 
For  example,  if  a  date  of  February  20,  1976,  were  chosen,  then  only  medical  records 
with  entries  later  than  that  date  would  be  included  in  the  sample.    The  evidence 
suggests  that  the  sample  procedures  were  implemented,  but  with  some  variability 
in  collecting  the  100  records  and  then  choosing  the  20  audit  cases.    This  varia- 
bility dealt  with  the  time-frame  that  residents  use  to  choose  records.  Some 
residents  had  to  increase  the  time-frame  in  order  to  collect  the  proper  number 
of  records;  thus  some  records  examined  had  longer  histories.    The  residents  in 
each  clinic  are  responsible  for  supervising  the  sampling.    Once  the  data  were 
collected  and  compiled,  the  Peer  Review  Committee  summarized  the  results  for  each 
clinic  and  a  percentage  of  criteria  items  present  were  summarized  in  table  form. 

During  the  final  site  visit,  the  audits  for  all  four  chronic  conditions 
were  complete,  and  audits  of  the  three  acute  conditions  were  being  completed. 
Results  of  the  audits  were  being  discussed  by  the  Peer  Review  Committee,  but 
no  specific  action  taken  based  on  these  results  could  be  identified. 

HCNHC  personnel  indicated  that  each  clinic's  Physician  in  Chief  (PIC)  was 
responsible  for  taking  action  based  on  audit  results.    The  Peer  Review  Committee 
had  the  responsibility  of  reporting  audit  results  to  the  PIC,  but  it  was  his 
decision  to  implement  educational  and  corrective  actions.    This  interaction  is 
somewhat  complicated,  since  all  physicians  work  through  the  DCM;  therefore  normal 
supervisory  relationships  come  into  play,  regardless  of  whether  audit  results  are 
the  basis  of  action. 


ADDITIONAL  COMMENTS 

One  area  of  contention  voiced  by  HCNHC  personnel  interviewed  was  the  amount 
of  judgment  required,  by  the  auditor,  to  decide  whether  specific  criteria  items 
had  been  completed  in  the  management  of  the  conditions.    Documentation  indicated 
that  problems  in  the  1975  audit  surfaced  regarding  the  "inflexibility  of  the 
standards."    Although  the  standards  were  explicit,  this  indicates  that  measure- 
ment is  still  difficult  when  relating  medical  record  information  to  established 
criteria.    Further,  when  judgment  is  required  by  the  auditors,  and  seven  auditors 
are  collecting  data  and  making  judgments,  the  reliability  of  the  data  is  ques- 
tionable.   HCNHC  personnel  recognize  these  issues  and  attempt  to  reduce  their 
impact  by  having  a  fairly  close  abstracting  and  monitoring  system.    As  each 
HCNHC  audit  developed,  DCM  physicians  have  attempted  to  identify  and  solve  review 
problems  prior  to  instituting  the  next  audit. 
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SECTION  III 


DELIVERY     SYSTEM  CHARACTERISTICS 


Harris  County  Neighborhood  Health  Clinics  deliver  services  to  approximately 
100,000  county  residents  per  year,  with  the  number  of  visits  exceeding  140,000 
and  the  number  of  encounters  approaching  250,000.    About  70%  per  year  of  these 
encounters  are  adults  and  30%  are  children.    The  Acres  Homes,  Baytown,  Ripley 
and  West  End  centers  see  about  10,000  patients  per  year,  while  the  Casa  de 
Amigos,  Settegast  and  Sunnyside  centers  see  approximately  20,000.    The  average 
number  of  patients  seen  per  day  for  all  clinics  is  561. 

The  HCNHC  is  supported  by  county  tax  revenues.    Some  funding  is  derived 
from  other  third  party  payers. 

Operational  responsibilities  for  the  seven  health  clinics  are  shared  by  the 
associate  administrator  of  the  Harris  County  Hospital  District  and  the  chairman 
of  the  Baylor  Department  of  Community  Medicine.    The  administrator  is  responsible 
for  the  fiscal  and  administrative  personnel  management  of  the  centers,  while 
the  chairman  is  responsible  for  the  management  of  medical  services  and  physicians. 
Each  clinic  is  operated  by  a  clinic  manager  who  controls  the  entire  administrative 
staff  and  reports  directly  to  deputies  of  the  administrator. 

Problem  Oriented  Medical  Records  (P0MR)  are  used  at  each  clinic.  This 
record  system  is  the  basis  of  the  Health-Illness  Profile,  which  was  developed  and 
is  presently  managed  by  the  DCM.    The  profile  has  four  basic  components:    (a)  iden- 
tification, (b)  health  profile,  (c)  illness  profile  and  (d)  services  rendered. 
Information  for  these  components  is  gathered  from  clinical  material  generated 
by  physicians  and  initial  working  data. 

The  seven  clinics  of  the  HCNHC  are  all  located  in  lower  socio-economic 
areas  of  Houston,  offering  good  access  to  patients  who  use  county  services.  The 
clinics  themselves  are  housed  in  diverse  facilities.    They  range  from  old  store- 
front buildings  to  converted  school  buildings  and  new  facilities  designed  expressly 
for  housing  a  medical  clinic. 


210 


SECTION  IV 


HCMS     STUDY  INFORMATION 


The  chairman  of  the  Department  of  Community  Medicine  was  initially  contacted 
via  correspondence  (October  8,  1975)  to  ascertain  interest  in  participating  in 
the  study.    Study  staff  initially  met  with  him  on  December  31,  1975,  whereupon 
agreement  for  participation  was  concluded.    Study  staff  also  met  with  the  present 
and  former  chairmen  of  the  Peer  Review  Committee  to  discuss  past  and  future  audit 
activities . 

After  these  discussions,  site  visits  were  made  to  five  centers  on  May  13 
and  14,  1976.    Prior  to  and  during  the  site  visits,  the  following  documents  des- 
cribing portions  of  the  quality  review  system  were  obtained: 

1.  A  System  for  a  Neighborhood  Health  Center  Peer  Review  -  A  1972  document 
describing  a  review  system  developed  and  implemented  at  the  Casa  de 
Amigos  Clinic  by  the  Department  of  Community  Medicine 

2.  Evaluation  of  Ambulatory  Care  in  Neighborhood  Clinics  -  A  short  paper 
describing  the  1975  review  procedures  at  the  HCNHC 

3.  The  1976  Objectives  for  the  Casa  de  Amigos  Clinic 

4.  HCNHC  Health-Illness  Profile 

5.  Casa  de  Amigos  monthly  report,  11/75 

6.  HCNHC  Statistical  Summary,  11/75 

7.  The  Harris  County  Hospital  District  Clinics:    A  Peer  Review,  5/74 

8.  Drug  Profile  1975 

During  the  site  visits  the  following  people  were  interviewed: 

1.  Physician-in-Charge  of  Casa  de  Amigos  and  chairman  of  the  Peer  Review 
Committee 

2.  Physician-in-Charge  of  Sunnyside,  member  of  the  Peer  Review  Committee 

3.  Physician  member  of  the  Peer  Review  Committee 

4.  Physician-in-Charge  of  Settegast  Clinic  and  member  of  the  Peer  Review 
Committee 

5.  Four  physician  residents 

During  the  site  interviews,  the  topics  included  the  following: 

1 .  History  of  the  HCNHC 

2.  The  role  of  the  Department  of  Community  Medicine,  Baylor  College  of 
Medici  ne 

3.  Previous  review  efforts 

4.  Current  review  efforts  and  methods 

5.  The  function  of  the  Peer  Review  Committee 

6.  Problems  with  the  current  review  procedures 
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HOUGH-NORWOOD  FAMILY  HEALTH  CENTER 


Cleveland,  Ohio 


SECTION  I 
INTRODUCTION 


Hough-Norwood  Family  Health  Center  (HNFHC)  is  a  nonprofit  neighborhood 
health  center  providing  ambulatory  services  in  Adult  Medicine,  Pediatrics, 
Dentistry  and  Mental  Health,  as  well  as  other  primary  care  services.  Hough- 
Norwood  serves  a  particular  geographic  area  in  Cleveland,  Ohio,  with  a 
population  consisting  of  predominantly  indigent  individuals  and  families. 
The  center  receives  most  support  funding  from  the  Bureau  of  Community  Health 
Services  (BCHS  -  DHEW)  but  has  limited  reimbursement  from  Titles  XVIII  and 
XIX,  and  county  welfare  support. 

There  are  approximately  27,000  active  medical  records  at  Hough-Norwood, 
which  has  a  medical  staff  of  30  physicians,  14  of  whom  are  full-time. 

In  terms  of  organizational  structure,  the  HNFHC  is  administered  by 
Cleveland  Neighborhood  Health  Services,  Inc.,  (CNHS)  which  in  turn  is 
directed  by  an  elected  board  of  trustees.    The  executive^ director  of  Hough- 
Norwood  reports  directly  to  CNHS,  and  is  responsible  for  daily  activities. 

The  executive  director  supervises  the  associate  director  for  health 
services  (the  medical  director),  who  manages  daily  medical  operations  of 
all  health  services,  and  the  associate  director  of  operations,  who  manages 
the  planning,  personnel,  fiscal  and  business  offices,  volunteer  services  and 
medical  records. 

The  medical  director  reports  directly  to  the  executive  director. 
Clinical  department  heads,  health  team  administrators  and  the  assistant 
director  of  health  services  report  directly  to  the  medical  director. 

Quality  assurance  activities  were  implemented  in  1969,  two  years  after 
Hough-Norwood  began  operations.    The  major  method  of  review  is  medical 
record  audit  performed  by  physicians  using  established  guidelines.  The 
goal  of  the  audit  is  to  identify  deficiencies  in  use  of  the  Problem  Oriented 
Medical  Record  (POMR)  and  to  identify  problem  areas  in  delivery  of  medical 
care. 

HNFHC  was  selected  to  participate  in  this  study  because  of  its  long 
term  experience  with  an  implemented  quality  assurance  program  and  its  use 
of  the  POMR. 


s 
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SECTION  II 


QUALITY       ASSURANCE  PROGRAM 


This  section  describes  quality  assurance  activites  at  Hough-Norwood 
Family  Health  Care  Center  in  terms  of  background,  documented  program  plans, 
and  operational  program  elements. 


BACKGROUND 

During  the  site  visit  interviews  for  this  study,  Hough-Norwood  personnel 
indicated  that  commitment  on  the  part  of  key  individuals  to  formal  quality 
assurance  activities  has  insured  a  consistent  level  of  operation  for  review 
activities.    Since  implementation  of  POMR,  emphasis  has  been  on  documenta- 
tion of  provider  performance  by  auditing  performance  relative  to  established 
audit  guidelines  and  standards  of  care.    Because  Hough-Norwood  serves  a 
medically  indigent  population  and  is  supported  by  government  financing, 
quality  assurance  activities  have  always  been  part  of  operations. 

Peer  review  activities  have  been  in  operation  since  1969.    Although  there 
have  been  several  revisions  in  the  program,  the  general  goal  has  remained 
unchanged,  which  is  to  be  a  "mechanism  for  physicians  to  assist  each  other 
in  improving  the  quality  of  medical  care  provided." 

The  quality  review  decision-making  lies  in  three  basic  areas: 

1.  With  physicians  -  each  physician  audits  three  charts  monthly 

2.  With  the  Audit  Committee  -  responsible  for  program  policy 
decisions  such  as  determining  the  direction  of  the  entire  peer 
review  program  and  its  implications 

3.  With  the  medical  director,  executive  director  and  department 
heads  -  responsible  for  major  policy  decision  making 


FORMALLY  PLANNED  ACTIVITIES 

Hough-Norwood's  planned  quality  assurance  activities  are  designed  to 
(a)  audit  the  use  of  the  problem  oriented  medical  record  (POMR),  (b)  identify 
problem  areas  in  patient  care,  (c)  audit  physician  adherence  to  Center  stan- 
dards.   Documents  note  that  two  types  of  audits  were  planned:    peer  audit  and 
checklist  audit. 

For  the  peer  audit  system,  the  documents  indicate  the  program  goal  is  to 
audit  three  records  per  physician  each  month,  according  to  an  explicit  audit 
format  used  by  each  reviewing  physician. 

The  five-page  audit  form  requires  the  following  specific  information  for 
each  review: 
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a.  Name  of  auditing  physician 

b.  Name  of  treating  physician 

c.  Patient  name  and  number 

d.  Date  of  audit 

e.  Date  of  complete  workup 

f.  Audit  number 

For  each  section  of  the  POMR,  the  audit  form  stipulates  several  questions 
the  reviewer  must  answer  using  the  information  in  the  record.    The  major 
sections  of  the  audit  form  include,  (a)  problem  list  (five  questions),  (b) 
list  of  plans  on  face  sheet  (four  questions),  (c)  progress  notes  (seven 
questions)  and  (d)  summary  which,  in  addition  to  providing  a  space  for  general 
comments,  requires  the  reviewer  to  stipulate  "urgent  needs"  or  "nonurgent 
needs."    The  auditing  physician  must  then  sign  the  audit  form  and  forward 
the  review  to  the  chief  of  services.    The  audited  physician  also  receives  a 
copy.    The  audit  system  is  designed  to  incorporate  professional  judgment 
in  the  audit  procedure,  and  although  specific  questions  are  outlined,  many 
of  the  questions  require  interpretation  of  medical  record  data  or  commentary 
on  medical  processes. 

When  unresolved  urgent  problems  are  noted  by  the  auditor,  the  record 
room  audit  supervisor  will  call  them  to  the  attention  of  the  audited  physician. 
A  "tickler  file"  is  established  to  insure  improvements  are  made.    When  the 
audited  physician's  chief  of  services  receives  and  reviews  the  audit  results, 
he  forwards  them  to  the  medical  director. 

After  an  audit  has  been  reviewed  by  the  chief  of  services  the  original 
audit  form  is  filed  in  the  physician's  permanent  record. 

Documents  indicate  charts  are  selected  for  audits  from  daily  appointment 
lists  by  the  medical  room  audit  supervisor.    Records  are  chosen  only  for 
patients  with  five  or  more  physician  encounters;  one  medical  record  is  chosen 
for  three  weeks  out  of  the  month.    Each  year  a  report  summarizing  the  HNHFC 
audit  activities  is  written  by  the  Audit  Committee  and  is  sent  to  the  exec- 
utive director  who  presents  audit  results  to  the  board  of  trustees  of  CNHS, 
Inc.,  at  their  annual  meeting. 

Specific  audit  instructions  are  noted  for  each  reviewing  physician. 
The  instructions  are  twofold: 

1.  Read  entire  chart  prior  to  answering  any  audit  questions,  to 
familiarize  yourself  with  the  patient's  history. 

2.  Answer  specific  audit  questions,  remembering  to  be  careful  and 
concerned  in  auditing.    A  careful  audit  will  be  more  helpful  to 
the  audited  physician.    Further,  the  instruction  urges  reviewing 
physicians  to  be  honest,  fair  and  complete. 

Finally,  the  documents  outline  basic  standards  for  the  POMR  which 
should  be  considered  by  the  reviewing  physician.    These  standards  (developed 
by  Lawrence  Weed,  M.D.,  in  1969)  are  incorporated  into  the  audit  format 
described  above. 
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The  checklist  audit  is  strictly  a  clerical  audit  of  the  POMR  conducted 
on  a  much  less  frequent  basis  than  the  peer  audit.    The  format  for  conducting 
this  audit  requires  the  following  data  elements: 

a.  Checklist  audit  number 

b.  Date  of  audit 

c.  Auditor's  name 

d.  Chart  number 

e.  Patient's  name 

f.  Enrollment  date 

g.  Physician's  name(s) 

The  audit  sheet  is  a  four-page  form,  requiring  the  auditor  to  check 
information  for  each  question  and  indicate  "Yes,"  "No"  or  "Not  Applicable." 
The  auditor  can  also  include  comments  if  necessary.    The  auditor  must  address 
a  total  of  57  POMR  requirements. 

There  are  no  documented  guidelines  for  the  clerical  person  to  follow  with 
this  audit,  though  the  record  room  supervisor  is  responsible  for  efficiency 
of  the  audits. 


OPERATIONAL  ACTIVITIES 

This  section  will  describe  the  operation  of  the  audit  procedures  outlined 
in  the  preceding  section.    The  two  sources  of  information  on  operations  were: 

1 .  On-site  interviews 

2.  Abstraction  of  Audit  Committee  minutes 

The  main  quality  assurance  activity  in  operation  is  physician  audit 
of  medical  records,  the  purpose  of  which  is  to  insure  that  physicians  and 
ancillary  services  are  using  the  POMR  properly,  thereby  reflecting  a  good 
level  of  care.    HNFHC  personnel  emphasized  the  importance  of  the  POMR  in 
facilitating  the  peer  audit,  which  has  been  implemented  generally  as  outlined 
in  HNFHC  documents. 

The  peer  audit  looks  primarily  at  physician  performance,  rather  than 
disease  or  diagnosis,  although  diseases  have  been  audited  in  the  past  when 
specific  problems  were  identified. 

In  the  peer  audit  procedure,  the  medical  director's  secretary  examines 
the  appointment  list  and  selects  names  on  patient  charts  for  audit.  Chart 
selection  is  guided  by  consideration  of  these  elements:    (a)  physician, 
(b)  patient's  recent  use  of  services  and  (c)  whether  the  record  had  been 
recently  audited.    There  is  no  systematic  or  random  selection.    Only  charts 
of  patients  being  seen  on  a  regular  basis  are  reviewed,  for  two  reasons: 

1.  To  examine  the  medical  care  provided  to  active  patients 

2.  To  examine  recent  physician  performances 

The  secretary  gives  the  list  to  the  record  room  supervisor  who  has  the 
charts  pulled.  Once  the  record  room  clerks  have  obtained  the  charts,  they 
consult  a  list  of  those  physicians  available  to  conduct  audits,  and  route 
the  records  to  them  with  a  peer  audit  form  attached. 
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Once  the  medical  record  is  delivered  to  the  auditor,  medical  record 
room  personnel  monitor  the  whereabouts  and  time  for  each  by  compiling  a  list 
of  active  chart  audits.    Once  the  physician  completes  the  audit,  the  record 
is  returned  to  the  record  room  where  it  is  refiled,  unless  some  urgent  problem 
has  been  identified.    If,  on  the  other  hand,  an  urgent  problem  has  been  noted 
by  the  auditor,  the  record  room  will  notify  the  audited  physician,  as  well  as 
the  medical  director.    Urgent  problems  are  supposed  to  be  dealt  with  immedi- 
ately. 

Completed  audit  forms  are  then  sent  to  the  chief  of  the  clinical  depart- 
ment of  the  audited  physician.    The  clinical  chief  reviews  and  signs  the 
audits.    Audit  forms  are  then  sent  to  the  medical  director,  who  reviews  and 
initials  them.    A  copy  is  sent  to  the  audited  physician  and  the  original  is 
filed  in  the  physician's  audit  file. 

If  a  significant  peer  audit  problem  is  discovered,  the  medical  director 
and  the  department  chief  are  responsible  for  deciding  a  course  of  action. 
They  usually  discuss  the  situation  in  the  context  of  what  type  of  feedback  a 
chief  of  service  should  initiate,  since  he  is  direct  supervisor  of  the 
physicians. 

The  peer  audit  process  has  been  implemented  as  outlined  in  Hough-Norwood's 
plans.    Although  the  audit  form  has  been  altered  several  times  over  the  years, 
the  audit  procedures  have  remained  substantially  the  same.  Hough-Norwood 
personnel  indicated  the  audits  usually  require  20  to  30  minutes  to  complete, 
although  the  time  varies  by  provider. 

Audit  Committee  meeting  minutes  spanning  a  seven-month  period  (July, 
1975  to  January,  1976)  were  reviewed  to  substantiate  the  operational  level  of 
the  peer  audit.    Each  meeting  was  attended  by  all  committee  members:    (a)  dental 
director,  (b)  medical  records  administrator,  (c)  medical  director,  (d)  chief, 
adult  medicine,  (e)  chief,  pediatrics  and  (f)  executive  director.  This 
committee,  chaired  by  the  medical  director,  reports  directly  to  the  executive 
director. 

In  two  meetings  in  July,  1975,  Audit  Committee  discussions  concerned 
two  main  areas: 

1.  Revision  of  the  five-page  peer  audit  form 

2.  Method  for  compiling  audit  results 

Although  no  specific  audits  were  discussed,  the  meetings  dealt  with  cri- 
tical administrative  components  of  the  audit  program. 

Operational  activities  not  identified  in  interviews  included  a  general 
tally  of  audit  results  compiled  after  results  have  been  forwarded  to  the  medi- 
cal director's  office.    Each  physician  has  an  individual  tally  sheet  which 
allows  compilation  of  audit  results  in  columns  to  identify  problem  areas 
for  individual  physicians. 

The  September,  1975,  Audit  Committee  meeting  minutes  noted  several  points: 
(a)  as  of  August,  1975,  977  audits  had  been  performed,  (b)  audits  should  be 
restricted  to  patients  with  at  least  five  visits,  (c)  responsibility  for 
audit  follow-up  was  delegated  to  three  committee  members. 
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The  January,  1976  meeting  noted  the  following  quantities  of  peer  audits: 

October        45  records  selected  for  audit,  20  completed 

November  --  61  records  selected  for  audit,  53  completed 

December  --  51  records  selected  for  audit,  14  completed 

January    --  29  records  selected  for  audit,  1  completed 

This  information  documents  a  high  level  of  audit  activity,  although  the 
activity  is  variable.    Hough-Norwood  personnel  said  this  inconsistent  per- 
formance was  due  to  the  changing  demands  placed  on  physicians,  and  that  audit 
activity  is  reduced  when  the  workload  increases. 

In  general,  the  minutes  reflect  a  high  level  of  activity  for  the  peer 
audit  process.    The  minutes  also  indicate  meeting  discussions  center  not 
only  on  audit  procedure  issues,  but  also  on  operation  of  the  total  health 
program.    The  committee  never  conducts  audits,  but  defines  policies  and 
procedures  for  the  audit  program.    The  minutes  did  not  reflect  any  discussion 
of  action  or  feedback  for  specific  physicians.    These  processes  are  conducted 
outside  committee  settings  among  the  medical  director,  department  chief  and 
individual  physicians.    The  minutes  did  reflect  specific  discussions  aimed 
at  solving  problems  identified  through  the  audit. 

The  operational  status  of  the  checklist  audit  was  nil.    Audit  Committee 
minutes  reflect  discussions  about  the  value  of  this  audit.    One  specific 
operational  activity  was  a  revision  of  the  checklist  audit  form  procedure 
by  the  medical  records  room  supervisor. 

During  the  interviews,  HNFHC  personnel  indicated  this  audit  is  nonfunc- 
tional, and  when  it  was  operational,  it  was  done  less  frequently  than  the 
peer  audit.    One  major  problem  in  implementing  this  audit  was  lack  of  medical 
record  room  personnel  and  time  for  conducting  clerical  audit. 


ADDITIONAL  COMMENTS 

The  operational  activity  of  HNFHC  audits  is  generally  high,  although  it 
seems  to  vary  with  the  amount  of  volunteer  time  providers  can  devote  to 
audit  activities. 

Audit  procedures  are  discussed  continually  in  terms  of  productivity 
and  revision.    The  program  seems  amenable  to  input  from  physician  and  admini- 
strative staff,  as  well  as  outside  sources. 

Two  Hough-Norwood  personnel  have  conducted  external  audits  of  other 
programs  for  the  State  of  Ohio  Department  of  Health  and  the  Albert  Einstein 
Medical  School  Audit  Group.    They  are  familiar  with  quality  assurance  methods 
and  literature.    The  major  impetus  behind  the  audit  program  is  the  enthusiasm 
the  medical  director  and  the  executive  director  bring  to  the  program;  equally 
important  is  their  knowledge  of  their  own  program  and  the  delivery  system, 
allowing  identifiable  objectives  and  accomplishments. 
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SECTION  III 


DELIVERY       SYSTEM  CHARACTERISTICS 


As  a  neighborhood  health  center,  Hough-Norwood  Family  Health  Center 
offers  medical,  dental,  optometric,  specialty,  mental  health,  alcohol 
treatment,  drug  abuse  treatment,  x-ray  and  laboratory  services.  To  pro- 
vide these  services,  the  medical  staff  is  divided  into  five  teams.  Each 
team  area  has  a  reception  office,  pediatric  and  adult  examination  rooms, 
dental  suite  and  medical  and  dental  offices. 

Other  support  services  provided  through  HNFHC  are  pharmacy,  trans- 
portation, night  telephone  services,  special  maternal  and  infant  projects 
for  obstetrical  care  and  patient  education  services  and  family  planning. 
Most  maternity  patients  are  referred  to  the  Metropolitan  General  Hospital 
for  delivery. 

The  center  also  has  an  indoor  nursery  and  outdoor  playground  for 
children  whose  family  members  must  attend  the  clinic.    Social  services  are 
provided  and  some  individual  and  family  counseling  services  are  available. 

HNFHC  has  approximately  27,000  patients  with  active  charts.    The  popu- 
lation is  predominantly  black  and  lives  within  a  five-mile  radius  of  the 
health  center. 


AFFILIATIONS 

Hough-Norwood  is  a  nonprofit  corporation  accredited  by  the  State  of 
Ohio  as  an  ambulatory  care  facility  and  as  a  Joint  Commission  for  Alcohol 
Treatment  Program. 

It  has  formal  affiliations  with  Mount  Sinai  Hospital,  University  Hos- 
pital and  Cleveland  Metropolitan  General  Hospital  in  terms  of  backup  facili- 
ties.   There  are  informal  relationships  with  state  and  local  medical  associa- 
tions . 


FUNDING 

As  stated  previously,  Hough-Norwood  receives  funding  from  various  federal, 
state  and  local  agencies  as  follows: 

80%  Federal 
16%  State 
4%  Local 

From  September  1,  1975,  thru  August  31,  1976,  the  total  budget  was 
$4,906,745.    Of  that  amount  $2,658,532  went  to  personnel  and  direct  care, 
15%  to  supportive  action  and  25%  to  general  services. 
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The  project  has  been  supported  by  grants  from  the  Bureau  of  Community 
Health  Services.    There  has  also  been  limited  reimbursement  from  Titles 
XVIII  and  XIX  and  county  welfare  (General  Relief). 


ORGANIZATIONAL  STRUCTURE 

This  subject  was  partially  covered  in  the  introduction  to  this  paper. 
The  administrative  staff  at  HNFHC  is  composed  of  55  persons  and  the  medical 
staff  is  composed  of  30  salaried  physicians.    Of  these  30,  approximately 
half  work  less  than  16  hours  per  week;  7.7  are  full-time  equivalent  physi- 
cians in  medicine  and  6.3  are  in  pediatrics. 

When  a  physician  joins  the  health  center  he  or  she  signs  a  contract. 
The  contract  in  essence  says  that  the  physician  will  work  in  accord  with  the 
medical  director,  including  in  quality  assurance  activities. 


MEDICAL  RECORD 

Hough-Norwood  utilizes  a  POMR  which  is  also  a  unit  record,  incorporating 
all  entries  by  all  health  providers  into  a  single  record. 

The  data  base  for  the  medical  record  is  collected  at  the  first  appoint- 
ment visit.    This  is  called  base-line  screening.    During  this  visit  all 
routine  laboratory  exams,  X-rays,  electrocardiograms,  and  dental,  visual  and 
audiological  screening  tests  are  performed  and  entered  in  the  medical  record. 
During  the  second  patient  visit,  a  complete  physical  examination  is  done. 
From  this  examination  the  problem  list  is  generated. 

The  medical  record  unit  number  is  comprised  of  six  digits  per  family 
plus  two  suffix  digits  which  identify  the  family's  physician. 

Through  the  use  of  POMR  baseline  data,  paramedical  personnel  initiate 
the  follow-up  of  individual  patients.    Then  the  physicians  proceed  with 
physical  examinations.    All  visits  are  coded  (ICDA-8,  International 
Classification  of  Diseases,  Adapted,  or  CPT,  Current  Procedure  Terminology) 
for  billing  purposes  prior  to  returning  the  record  to  the  medical  record 
department. 


COMPUTER  SYSTEM 

A  computer  system  is  used  at  HNFHC  only  for  billing  purposes.  Every 
patient  encounter  is  recorded  on  an  Encounter  Form  and  the  information  is 
entered  into  the  computer  at  the  billing  office. 

The  billing  process  is  as  follows: 

1.  The  patient  is  seen  by  a  physician 

2.  An  encounter  form  is  attached  to  the  medical  record 

3.  The  medical  record  is  taken  to  the  coding  and  billing  unit  after 
the  physician  has  seen  the  patient 
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4.  The  medical  record  is  coded  ICDA  (ADA,  American  Dental  Associa- 
tion, codes  are  entered  by  dental  personnel) 

5.  The  encounter  form  is  checked  and  ICDA  codes  are  added,  or  both 

6.  Data  from  the  encounter  form  is  entered  into  a  computer  system 

7.  Bills  are  produced  from  the  information  in  the  computer  and  then 
a  "hard  copy"  is  graded  based  on  the  service  and  organization  to 
be  billed 


PHYSICAL  LAYOUT 

HNFHC  has  two  service  area  locations.  One  houses  three  teams  and  the 
other  two.  Ancillary  services  are  present  on  both.  The  two  locations  are 
designed  to  serve  35,000  patients. 
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SECTION  IV 

H  CMS       STUDY  INFORMATION 


Hough-Norwood  Family  Health  Center  has  participated  in  several  quality 
assurance  implementation  projects,  in  which  different  research  teams  imple- 
ment certain  methods  (e.g.,  baseline  adult,  staging)  to  gain  operational 
experience.    They  agreed  to  participate  in  this  study  by  correspondence  dated 
January  8,  1976. 

Prior  to  the  site  visit  HNFHC  supplied  HCMS  staff  with  the  following 
documentation: 

1.  General  descriptive  manual  of  all  aspects  of  the  Hough-Norwood 
Clinic  (1965) 

2.  Registration  forecast  for  both  Hough-Norwood  settings 

3.  Budget  outline 

4.  Example  of  complete  medical  record 

5.  Encounter  forms 

6.  Minutes  for  Audit  Committee  (July,  1975  through  January,  1976) 

7.  Set  of  tabulations  indicating  compliance  with  Diagnostic  and 
Therapeutic  Center  Standards  (1969  to  1975) 

8.  Hough-Norwood  organizational  chart  (1975  to  1976) 

9.  List  of  the  Board  of  Trustees  and  each  member  in  the  organization 
(1975  to  1976) 

10.  A  Review  of  the  Quality  of  Medical  Care  at  Hough-Norwood  Family 
Health  Care  Center  conducted  by  the  Evaluation  Unit,  Department 
of  Community  Health,  Albert  Einstein  College  of  Medicine  (1970) 

11.  Billing  forms  for  various  services 

12.  Suggested  standards  to  be  used  in  physician's  audit  of  the 
POMR  by  Lawrence  B.  Weed  (April  1969) 

13.  Physician's  audit  forms  and  procedures 

14.  Rate  of  appointments  kept  September  21  to  October  3,  1970,  and 
November,  1971 

The  documentation  provided  a  good  description  of  the  Hough-Norwood 
delivery  system.    At  the  site  visit  staff  concentrated  on  collection  of 
data  on  the  present  level  and  type  of  quality  assessment  activities  being 
conducted  at  the  center. 

Four  persons  were  interviewed  during  the  site  visit,  the  medical 
director,  the  executive  director  and  two  medical  record  room  personnel. 

During  the  interviews,  several  specific  subjects  were  discussed: 

1.  Evolution  of  peer  audit  activities 

2.  Implementation  of  POMR  and  its  use  in  audit  procedures 

3.  Administrative  structure  of  the  quality  assurance  program, 
including  direct  participation  by  physicians  and  administrative 
staff  support 

4.  Duties  of  personnel  involved 
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5.  Conceptual  framework  of  audit  procedures  at  Hough-Norwood 

6.  Specific  elements  of  review-audit  process 

7.  Elements  within  delivery  setting  which  affect  the  review-audit 
process 

8.  Data  flow  of  completed  audits 

9.  Basis  of  decision-making  in  review-audit  process 

10.    Development  and  use  of  medical  care  standards  in  review-audit 
process 

Most  of  the  interview  time  was  spent  with  the  medical  director  and  the 
executive  director.    The  medical  director  has  full  responsibility  for  the 
development  and  implementation  of  all  periodic  audit  procedures  at  HNFHC, 
as  well  as  all  clinical  activities  and  the  review  of  those  activities.  The 
executive  director  oversees  all  fiscal  and  service  components  of  the  clinic. 
The  executive  director  has  very  little  direct  communication  with  the  medical 
staff,  but  interacts  with  the  medical  director  to  determine  the  status  of 
periodic  review  activities.    He  is  committed  to  administrative  and  budgetary 
support  of  quality  assurance  activities. 
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VINELAND  FAMILY  HEALTH  CENTER 


Vineland,  New  Jersey 


SECTION  I 
INTRODUCTION 


The  Vineland  Family  Health  Center  (VFHC)  in  Vineland,  New  Jersey, 
is  a  free-standing  health  maintenance  organization  (HMO)  offering 
primary  medical  care. 

VFHC  began  operation  in  April,  1974,  after  receiving  a  grant 
from  the  Bureau  of  Community  Health  Services  (DHEW),  and  was  awarded 
Certificate  of  Need  approval  from  the  New  Jersey  Department  of  Health 
in  August,  1974.    The  Certificate  of  Authority  under  New  Jersey's  HMO 
enabling  legislation  was  requested  and  received  in  1975.    VFHC  plans 
to  market  a  prepaid  benefit  package  over  the  next  24  to  36  months  to 
enlarge  subscriber  population  and  to  decrease  dependence  on  federal 
funding. 

VFHC  is  a  nonprofit  corporation  with  a  board  of  directors  of 
14  members;  a  project  director  (administrator),  a  medical  director, 
four  full  time  equivalent  (FTE)  physicians  and  five  ancillary  personnel. 
The  quality  assurance  program  is  entitled  "Monitoring  Quality  through 
Chart  Audit"  and  is  an  explicit  criteria  audit  of  specific  data 
elements  in  the  Problem  Oriented  Medical  Record  (POMR). 

The  Vineland  Center  was  included  in  the  study  since  it  repre- 
sented a  new  and  small  medical  care  setting.    HCMS  staff  received 
the  name  from  the  Association  of  New  York  Neighborhood  Health  Centers, 
where  VFHC  had  been  invited  to  present  a  description  of  their  quality 
assessment  program. 
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SECTION  II 

QUALITY     ASSURANCE  PROGRAM 


BACKGROUND  AND  FORMALLY  PLANNED  ACTIVITIES 


The  impetus  for  development  of  quality  assessment  activities  at 
VFHC  emanated  from  the  medical  director's  belief  that  a  prepaid  health 
plan  must  initiate  such  activities  to  assure  external  sources  (govern- 
ment, competition,  employer-employee,  etc.)  a  certain  level  of  quality 
in  medical  services.    The  medical  director  has  overall  responsibility 
for  quality  assurance  activity  and  reports  directly  to  the  project 
director.    Titled  "Monitoring  Quality  through  Chart  Audit,"  the  VFHC 
quality  assurance  program  is  an  explicit  criteria  audit  of  specific 
data  elements  in  the  Problem  Oriented  Medical  Record  (POMR). 

VFHC  documents  provided  to  the  investigator  an  outline  of  audit 
goals,  procedures  and  outcomes.    VFHC's  audit  method  requires  contracted 
physicians  to  sign  a  written  commitment  describing  the  type  of  patient 
care  to  be  provided.    Use  of  the  Problem  Oriented  Medical  Record 
requires  physicians  to  document  plans  for  care  after  a  patient's  initial 
visit.    The  Principles  of  Practice  require  physicians  to  assign  each 
patient  to  one  of  the  following  three  care  groups: 

1 .    Episodic  Care  Patients 

Patients  who  come  for  episodic  care  only;  physician  plans 
for  no  care  beyond  immediate  because: 


a.  Patient  ordinarily  sees  another  physician 

b.  Patient  recently  had  a  medical  work-up  elsewhere, 
has  no  desire  for  another  one  now,  and  does  not 
want  to  decide  when  he  would  like  another  one 

c.  Patient  is  a  transient 

d.  Patient  does  not  choose  to  return  for  a  more 
complete  work-up 

e.  Patient  does  not  wish  to  make  a  commitment  as  to 
when  he  will  return 


2.  Standard  Care  Patients 

Physician  feels  the  patient  needs  only  the  usual  medical 
history  and  physical  exam  along  with  the  usual  tests. 
Further  tests  or  referrals  would  depend  on  the  findings. 

3.  Comprehensive  Care  Patients 

In  addition  to  standard  care,  the  patient  needs  further 
care  in  the  behavioral  or  social  areas  of  health. 
Patient  will  be  interviewed  by  a  social  worker  and  the 
case  will  be  considered  by  a  health  care  team  at  a  formu- 
lation conference,  during  which  a  health  care  plan  will 
be  developed.    The  plan  will  then  be  presented  by  a 
physician  or  other  health  professional  to  the  family, 
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a  responsible  member  of  the  family,  or  the  patient 
himself,  for  acceptance  or  alteration.    The  patient 
will  be  called  for  reassessment  by  the  team  within 
12  months. 

Detailed  guidelines  for  the  three  levels  include  specific  data 
elements  which  must  be  in  the  chart  as  the  basis  for  audit.  Audits 
can  be  completed  by  clerical  personnel.    If  physicians  use  the  POMR 
properly,  audits  can  be  completed  from  information  on  the  problem 
sheet.    The  VFHC  program  is  based  on  the  premise  that  a  POMR  reflects 
a  particular  level  of  care;  therefore,  the  chart  audit  becomes  the 
central  quality  assurance  activity. 

The  program's  basic  data  collection  document  is  the  audit  sheet, 
consisting  of  15  questions  about  POMR  content  requiring  a  yes  or  no 
answer.    After  medical  records  are  audited,  the  audit  sheets  are  for- 
warded to  reviewed  physicians  concerned  with  corrections  and  explana- 
tions, then  to  the  medical  staff  meeting,  where  audits  and  other 
clinic  business  are  reviewed.    The  reverse  side  of  the  Audit  Sheet 
allows  for  "physician  comments  and  actions,"  along  with  "follow-up 
with  individual  physician,"  based  on  audit  results. 

Audits  may  be  done  on  charts  selected  by  a  particular  diagnosis. 
Explicit  management  criteria  for  these  diagnoses  are  developed  prior 
to  review  and  the  audit  is  conducted  by  the  entire  medical  staff. 


OPERATIONAL  ACTIVITIES 

The  operational  status  of  the  VFHC  audit  program,  as  of  February  23, 
197  6,  involved  the  implementation  of  eight  chart  audits  and  one 
diagnostic  audit. 

During  the  initial  period  of  clinic  operations  when  audit  pro- 
cedures were  implemented,  VFHC  staff  physicians  were  under  full 
work  loads.    The  medical  director  said  staff  has  little  time  (no  more 
than  "60  to  90  minutes  each  week")  for  medical  staff  meetings,  which 
include  not  only  discussing  general  medical  issues,  but  the  develop- 
ment of  criteria  for  management  and  in-service  education  as  well  as 
voluntary  audit  activities. 

The  medical  director  has  invested  significant  amounts  of  time  in 
developing  and  revising  the  principles  of  practice  for  VFHC,  organizing 
support  among  staff  physicians,  and  developing  the  Audit  Sheet.  The 
director  indicated  there  is  difficulty  in  persuading  physicians  to 
commit  themselves  to  regularly  scheduled  audits,  and  the  small  staff 
size  deters  development  of  formal  procedures. 

The  focus  of  VFHC  operational  audit  procedures  is  the  review  of 
a  sample  of  medical  records.    The  Medical  Records  Review  Committee 
consists  of  the  four  VFHC  physicians,  two  nurse  practitioners  and  one 
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health  assistant.    The  committee  meetings  are  chaired  by  the  medical 
director.    The  goal  of  the  committee  is  to  audit  a  small  sample  of 
medical  records  for  completeness  in  terms  of  specific  data  elements 
required  for  initial  health  evaluations. 

The  meetings  are  documented  on  a  tally  sheet,  which  collects 
the  following  information  for  each  case  reviewed: 

1 .  Date  of  service 

2.  Patient  name  and  identification  number 

3.  Name  of  provider 

4.  Whether  deficiencies  existed  (columns  to  mark  yes  or  no) 

5.  Whether  correction  was  recommended  (columns  to  mark  yes  or  no) 

6.  Follow-up  date,  if  any 

7.  Members  of  committee  present  at  the  meeting 

VFHC  provided  the  study  with  a  series  of  tally  sheets  to  document 
operational  audits,  but  did  not  provide  guidelines  governing  implemen- 
tation of  the  audits.    Since  the  audits  are  of  medical  records  with 
uniform  data  elements,  tally  sheets  note  the  medical  record  deficiencies 
found  during  audit. 

Eight  medical  record  audits  have  been  conducted  by  the  committee, 
from  June  12,  1975,  through  November  14,  1975.    These  audits  were  con- 
ducted during  committee  meetings  on  a  sample  of  recent  initial  health 
evaluations  pulled  by  medical  record  room  staff.    On  five  out  of  eight 
occasions  all  seven  committee  members  attended.    One  to  two  members 
were  absent  from  the  other  three  meetings. 

There  is  little  evidence  that  systematic  follow-up  procedures 
exist  to  correct  consistent  provider  errors,  based  on  recurring 
deficiencies  in  the  medical  record;  however,  follow-up  was  indicated 
in  eight  cases,  and  did  occur  during  the  first  two  meetings 
(June  12,  1975,  and  September  5,  1975). 

Forty- two  cases  were  reviewed  during  the  eight  committee  meetings. 
There  is  no  evidence  that  a  minimum  representative  sample  must  be 
pulled  to  conduct  audits  or  that  all  initial  health  evaluations  are 
reviewed.    Seven  providers  were  reviewed  over  the  six-month  period, 
with  one  provider  reviewed  13  times  (highest),  and  another  reviewed 
twice  (lowest). 

The  medical  record  room  staff  conducted  a  diagnosis-oriented  re- 
view in  December,  1975,  for  iron  deficiency  anemia.    Explicit  criteria 
were  developed  during  medical  staff  meetings,  with  major  input  coming 
from  one  physician  who  treated  most  of  the  cases.    The  medical  record 
room  supervisor  then  selected  17  cases  of  anemia  for  audit,  based  on 
the  explicit  criteria.    One  provider  was  reviewed  during  this  audit, 
which  sought  to  identify  deficiencies  in  the  process  of  care.    Out  of 
the  17  cases  audited,  no  deficiencies  were  reported.    This  was  the  only 
diagnosis  audit  conducted  as  of  the  site  visit  date. 
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The  medical  director  noted  that  the  level  of  quality  assurance 
activity  at  VFHC  will  fluctuate  over  the  first  two  years  of  operation 
because  of  the  various  problems  a  new  delivery  system  faces  in 
becoming  permanently  established  within  a  service  area. 


ADDITIONAL  COMMENTS 

During  the  interview,  the  medical  director  mentioned  several 
variables  which  seem  to  affect  the  implementation  of  formal  quality 
assurance  activities: 

1.  Recruitment  of  physicians  to  the  practice 

2.  Focus  of  VFHC  on  marketing  prepaid  package 

3.  Focus  of  VFHC  on  staying  financially  viable 

4.  No  interaction  between  provider  staff  and  administrative 
personnel  regarding  audit  activities 

5.  Small  population  base  which  does  not  generate  enough 
cases  for  review 

6.  Audit  methods  not  defined  well  enough,  because  of  inexperience 

7.  Reluctance  of  physicians  to  volunteer  time  to  audit  activities 
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SECTION  III 


DELIVERY     SYSTEM  CHARACTERISTICS 


VFHC  is  a  small  primary  medical  group  practice  providing  out- 
patient care  to  an  enrolled  population.    VFHC  attempts  to  stress 
health  maintenance  activities  by  giving  each  enrol  lee  a  complete 
health  evaluation  during  the  initial  visit.    The  following  benefits 
are  provided: 

1.  Routine  office  visits 

2.  Physical  examination  and  immunizations 

3.  Family  health  nurse  visits 

4.  Consultations  and  specialty  referrals  (VFHC  health  center 
physician  must  authorize  a  referral  to  a  specialist) 

5.  Well  child  care 

6.  Eye  examinations,  refraction  and  hearing  tests 

7.  Health  education  and  family  planning  visits 

8.  Diagnostic  X-ray,  laboratory  and  pathology  tests 

9.  Therapeutic  radiology 

10.  Minor  surgery 

11.  Maternity  care  including  pre-  and  postnatal  visits 

12.  Anesthesia 

13.  Hemodialysis  treatment 

14.  Physical  therapy 

15.  Emergency  care  24  hours,  7  days  a  week 

During  the  first  operational  year,  VFHC's  distribution  of  patients 
by  source  of  payment  was  as  follows: 

46%  Private  pay  (self-pay,  Blue  Shield,  employee-paid ) 
32%  Medicaid 

4%  Medicare 
13%  Prepayment 

5%  Other 

VFHC  has  also  consummated  a  fiscal  and  marketing  affiliation  with 
New  Jersey  Blue  Cross.    Blue  Cross  is  in  the  process  of  offering  the 
HMO  option  as  an  alternative  to  the  present  service  benefit  coverage. 
Blue  Cross  enjoys  a  50%  market  penetration  in  VFHC's  service  area 
which  will  boost  VFHC's  marketing  plan. 

The  total  number  of  enrol  lees  from  small  business  employer-employee 
groups  is  approximately  3,600. 
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The  medical  audit  employed  at  VFHC  is  a  Problem  Oriented  Medical 
Record  (POMR).    Guidelines  for  use  of  the  POMR  by  VFHC  physicians 
are  described  in  the  Principles  of  Practice.    The  data  elements 
include: 

1.  Data  base,  which  includes  an  initial  evaluation  of  the 
patient  (medical  history,  physical  examination) 

2.  Documented  problems 

3.  Diagnosis  of  problems 

4.  Date  and  patient  information 

5.  Resolution  of  problem 

6.  Hospitalization,  referrals,  conferences,  telephone  calls 

7.  A  plan  to  address  particular  problems 

8.  Progress  notes  (free  entry  facts) 

9.  Flow  chart  to  document  each  stage  of  patient  evaluation 
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SECTION  IV 

HCMS     STUDY  INFORMATION 


HCMS  staff  contacted  VFHC  by  correspondence  on  November  21,  1975, 
to  ascertain  their  interest  in  the  study.    VFHC  indicated  willingness 
to  participate  in  correspondence  dated  December  5,  1975,  and  forwarded 
initial  documentation  to  study  staff  on  December  8,  1975.    The  site 
visit  was  conducted  February  23,  1976. 

Initial  documentation  of  VFHC  quality  assessment  components 
includes: 

1.  Principles  of  Practice  (revised,  August  1,  1975) 

2.  Correspondence  outlining  specific  elements  of  program 
(December  1,  1975,  269  pages) 

3.  Continuation  application  for  FY  1976  (March  1,  1975) 
which  included  the  following  major  documents: 

a.  Summary  report  to  Board  of  Directors 

b.  State  of  Affairs  (statement) 

c.  First  Operational  Year  Statistical  Summary 
(April ,  1974  to  March,  1975) 

d.  VFHC  Marketing  Plan  (117  pages) 

e.  VFHC  delivery  system  characteristics  (15  pages) 

f.  Medical  chart  documents  (8  attachments  to  report) 

g.  Budget  and  fiscal  diagrams  (10  pages) 

h.  Exhibits  (97  pages) 

The  site  visit  interviews  were  with  the  medical  director  and  the 
supervisor  of  the  chart  room.    The  Data  Collection  Instrument  (DCI) 
and  Administrative  Constraints  Questionnaire  (ACQ)  were  not  used 
extensively  in  the  interview  because  of  the  low  operational  frequency 
of  formal  quality  assessment. 

During  the  interview  several  topics  were  covered,  including: 

1.  Operational  status  of  VFHC 

2.  Development  and  operational  status  of  audit  procedures 

3.  Problems  in  developing  and  implementing  audit  procedures 

4.  Prospective  accomplishments  of  audit  programs 

5.  Specific  audit  completed  at  VFHC  for  one  identified 
probl em-diagnosis 

6.  Criteria  development 

7.  Historical  development  of  VFHC 

8.  Activities  of  medical  record  room  in  audits 

9.  Problems  faced  by  medical  director  in  establishing 
audit  program 

Study  staff  generally  concentrated  on  questions  dealing  with 
issues  in  developing  and  implementing  quality  assessment  activities 
in  a  relatively  new  medical  care  setting. 
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INDIAN  HEALTH  SERVICES 


Tucson,  Arizona 


SECTION  I 


INTRODUCTION 


The  Indian  Health  Service  (IHS)  in  Tucson,  Arizona,  is  one  of  several 
independent  medical  delivery  systems  providing  services  to  an  American 
Indian  population.    The  Tucson  IHS  employs  ten  physicians  and  five  medical 
assistants. 

The  Tucson  IHS  operates  a  branch  office  of  the  Office  of  Research 
and  Development  (ORD),  which  is  the  parent  organization  for  the  Sells  Service 
Unit,  the  Desert  Willows  Training  Center  (trains  tribal  personnel)  and  the 
Health  Program  System  Center.    There  are  a  number  of  distinct  offices 
within  ORD,  one  of  which,  the  Office  of  Planning  and  Coordination,  is 
responsible  for  coordinating  various  functions  and  units  within  ORD.  The 
Experimental  Medical  Care  Review  Organization  (EMCRO),  funded  by  the 
National  Center  for  Health  Services  Research,  is  part  of  the  Office  of 
Planning  and  Coordination.    The  overall  objective  of  ORD  is  to  study  and 
improve  the  effectiveness  and  accessibility  of  IHS's  medical  services. 
The  main  objective  of  the  EMCRO  project  was  to  develop,  test  and  implement 
various  methodologic  approaches  to  assessing  the  quality  of  medical 
services. 
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SECTION  II 


QUALITY       ASSURANCE  PROGRAM 


Prior  to  the  implementation  of  the  EMCRO  project,  some  ambulatory 
quality  review  activities  had  already  been  studied  and  implemented, 
establishing  a  methodologic  and  operational  foundation  for  the  EMCRO  project. 
Two  IHS  physicians  were  interested  in  developing  solutions  for  reaching 
and  treating  high  risk  population  groups  experiencing  prevalent  medical 
problems  on  the  reservation.    The  physicians  developed  a  "staging"  approach 
which  identified  sequential  phases  of  the  severity  of  a  health  problem. 
This  required  problem  identification  and  interaction  techniques  which 
corresponded  with  the  phases.    The  purpose  of  identifying  stages  was  to 
allocate  limited  medical  resources  to  attacking  the  problem  in  less  severe 
stages  to  prevent  the  problem  from  advancing  to  a  more  severe  and  costly 
stage. 

Since  medical  resources  are  particularly  scarce  in  these  settings, 
the  two  physicians  felt  it  necessary  to  develop  a  method  allowing 
tribal  health  employees  to  assist  in  identification  and  resolution  of 
problems.    One  physician  developed  a  set  of  protocols  for  the  particular 
disease  conditions,  which  were  then  placed  in  a  format  of  information 
gathering  forms.    These  forms  would  allow  a  worker  to  ask  a  series  of 
questions-    Based  on  patient-provided  answers,  the  worker  could  then 
place  the  patient  in  the  applicable  stage  of  disease  development.  This 
simultaneously  allowed  the  tribal  worker  to  stage  the  patient  and  refer 
to  assessment  criteria  in  order  to  evaluate  the  case  in  the  hope  of  leading 
to  successful  management  without  the  intervention  of  a  physician.  The 
combination  of  identifying  high  risk  population  groups  and  having  the 
ability  to  allocate  resources  in  focused  areas  would  potentially  make  the 
system  more  effective  and  cost  beneficial. 

The  ability  to  determine  risk  levels  for  specific  problems  was 
important  to  allow  changes  in  the  allocation  of  medical  resources.  When 
testing  began,  physicians  found  medical  resources  more  scarce  than  antici- 
pated, necessitating  the  use  of  fewer  field  health  workers.    This  merely 
accentuated  the  need  for  a  method  and  program  to  distribute  resources  to 
higher  risk  sections  of  the  population. 

One  physician  began  a  study  of  old  medical  records  in  order  to  deter- 
mine what  factors  could  be  used  to  predict  high  risk  groups.    Eleven  factors 
were  delineated  which  potentially  could  identify  a  high  risk  gastroenter- 
itis patient.    This  predictive  model  was  then  tested  to  document  its 
applicability.    The  physician  separated  newborn  infants  into  high  and  low 
risk  groups  based  on  the  eleven  information  factors.    This  information  was 
collected  on  the  day  an  infant  was  born.    Once  high  and  low  risk  infants 
were  identified,  they  were  separated  into  two  groups  with  both  high  and 
low  risk  infants: 
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(a)  Non-control  groups,  which  received  specifically  allocated 
preventive  services 

(b)  Control  groups,  which  received  no  preventive  services 

The  results  of  the  study  indicated  the  high  risk  group  which  received 
preventive  care  had  a  drop  in  serious  morbidity  (i.e.,  the  gastroenteritis 
did  not  advance  to  more  serious  stages).    Low  risk  groups  which  received 
preventive  services  experienced  essentially  no  difference  in  occurrence 
of  gastroenteritis. 

The  two  IHS  physicians  felt  they  had  developed  a  fairly  accurate 
predictive  model.    Another  study  was  conducted  dealing  with  infant  diarrhea, 
which  also  provided  supporting  evidence  for  this  type  of  predictive  model. 
From  these  studies,  three  key  points  regarding  implementation  of  quality 
assurance  methods  were  learned: 

1.  Standards  for  information  gathering  and  assessment  were 
necessary  and  could  be  successfully  developed 

2.  When  there  are  not  enough  medical  resources  to  deal  with 
everyone  and  all  problems,  then  developing  a  risk  model  is 
necessary  for  proper  allocation  of  scarce  resources. 

3.  Community-based  personnel  could  be  used  in  information 
gathering,  assessment  and  implementing  preventive  care 
procedures. 

The  IHS  physicians,  citing  this  experience  with  the  risk  model  and 
staging  methodology,  applied  for  an  Experimental  Medical  Care  Review 
Organization  (EMCRO)  grant  in  late  1972.    The  grant  was  awarded  in  July, 
1973,  for  a  three  year  period.    The  EMCRO  program  had  three  general 
phases: 

a.  Develop,  test  and  evaluate  ambulatory  medical  care  review 
methods  for  Sells  Service  Unit 

b.  Implement  programs  to  correct  problems  identified  through 
development  and  testing  of  methods 

c.  Measure  impact  of  program  activity  implemented  to  correct 
medical  system  deficiencies  in  addressing  ambulatory  care 
problems 
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EXPERIMENTAL  MEDICAL  CARE  REVIEW  ORGANIZATION  (EMCRO)  PROGRAM 


The  EMCRO  program  was  essentially  designed  to  develop  new  ambulatory 
review  methods.    The  purpose  was  to  collect  information  on  the  testing 
of  specific  methods  in  order  to  export  tested  and  hopefully  proven  methods 
for  use  in  other  medical  care  systems  with  similar  organizational  character- 
istics. 

The  method  developed  prior  to  the  EMCRO  grant  was  used  as  the  basis 
for  all  IHS  EMCRO  health  program  developments.    The  method  is  designed  to 
recognize  and  audit  specific  data  for  a  group  of  specific  health  problems 
(tracers)  and  to  measure  the  effectiveness  of  IHS  services  by  examining 
both  process  and  outcome  of  care.    This  is  accomplished  by  answering 
questions  (indicators)  relating  to  the  effectiveness  of  care.  Results 
are  compared  to  predetermined  levels  of  compliance  (performance  standards) 
to  judge  the  quality  of  care.    The  source  of  these  standards,  according 
to  documentation,  is  local  medical  staff.    The  approach  is  problem  oriented, 
in  that  all  identified  problems  must  be  prevalent  and  of  significant  risk 
to  the  population,  as  well  as  being  measurable  by  several  quality  assess- 
ment activities.    The  initial^ tracer  conditions  chosen  for  this  study  were 
gastroenteritis,  hypertension,  and  maternal  child  health  care.    This  method 
evolved  into  a  nine-step  approach  entitled  FOCUS  (Formulation  of  an  Operating 
Construct  ^Utilizing  Skills  for  health). 

Prior  to  the  implementation  of  the  nine-step  FOCUS  approach,  a  three- 
step  preaudit  process  was  performed  by  the  local  staff  as  follows: 

1.  Identify  material  which  is  to  be  audited.    Usually  charts  are 
selected  on  a  basis  of  demographic  information  or  diagnoses.  A 
sample  as  small  as  50  charts  is  adequate  to  measure  indicators. 
Larger  samples  are  needed  if  a  statistical  breakdown  of  results 
is  desired.    Two  instruments  are  used  for  data  collection, 

(a)  the  abstractor's  map,  which  is  a  branching  flow  diagram 
derived  from  a  clinical  criteria  set  which  prescribes  informa- 
tion to  be  collected  from  medical  records  and  allows  a  nonphysi- 
cian  to  make  a  reliable  interpretation  of  data  in  the  record,  and 

(b)  the  data  collection  form. 

2.  Length  of  study  period  is  clearly  defined.    Larger  time  frames 
will  provide  more  audit  specific  information,  but  if  the  study 
period  is  too  extensive,  specificity  is  diminished  and  the 
chance  for  rapid  performance  feedback  is  lost. 

3.  Auditing  the  charts.    Prior  to  the  audit,  trainees  employed 
for  data  collection  in  the  IHS  program  review  at  least  10 
charts  per  indicator  with  a  qualified  health  professional. 

Once  these  activities  had  been  considered  and  implemented,  the  IHS  EMCRO 
program  formally  began  implementing  the  nine-step  FOCUS  approach.  This 
approach  will  include  the  following  steps: 
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1.  Identify  a  high  priority  health  problem  or  health  area. 

2.  Define  the  health  problems  as  a  series  of  clinical  stages  such 
that  "stage  0"  represents  the  well  patient  risk,  and  "stage  n" 
represents  the  "ntn"  stage  of  the  problem.    Intervening  stages 
have  distinct  clinical  characteristics  with  respect  to  patho- 
logic and  functional  changes,  treatment  and  prognosis.  The 
stages  are  ranked  in  order  of  increasing  severity. 

3.  Identify  general  strategies  of  intervention,  based  on  what  stage 
should  be  given  the  most  emphasis.    The  objective  of  intervention 
is  to  promote  return  to  a  less  severe  stage  and  to  report  progres- 
sions to  a  more  severe  stage. 

4.  Identify  priority  groups  that  would  benefit  from  available 
resources. 

5.  Define  stage-specific  patient  criteria  knowing  those  elements 
of  care  necessary  to  solve  and  prevent  health  problems. 
Criteria  are  designed  to  specify  what  should  be  done  to  popula- 
tion group  based  on  frequency  or  duration  of  illness. 

6.  Translate  criteria  into  an  operational  structure  that  defines 
specific  duties  in  staff  supervision  of  patient  care. 

7.  Identify  health  personnel  training  requirements  required  for  the 
specific  problem  and  structure. 

8.  Develop  programs  to  evaluate  provider  performance,  the  continuity 
of  the  problem  solving  process  and  health  outcomes. 

9.  Implement  explicit  "feedback  mechanisms"  to  correct  the  weaknesses 
identified  through  the  evaluation. 

The  key  to  the  entire  method  is  the  use  of  protocols  by  nonprofes- 
sional health  personnel  for  collection  of  medical  data  which  is  subsequen- 
tially  used  in  the  evaluation  of  care.    Protocols  are  derived  from  explicit 
patient  care  criteria  established  for  system  users.    To  develop  criteria, 
it  is  necessary  to  look  at  the  total  problem  and  define  its  stages,  as 
indicated  above.    Critical  areas  of  treatment  should  be  defined  and  the 
following  questions  raised:    "What  are  the  necessary  conditions  for 
meeting  the  objectives  of  each  one  of  these  areas?"    The  criteria  should 
then  be  translated  into  indicators.    These  indicators  (questions)  are  data 
elements  which  will  show  whether  or  not  the  specific  procedure  has  been 
accomplished.    Each  criterion  is  associated  with  the  performance  standard 
and  an  indicator.    The  total  number  of  indicators  included  in  an  audit  is 
arbitrary.    Generally,  indicators  are  clustered  in  areas  where  correctable 
deficiencies  are  most  likely  to  exist. 

The  IHS  EMCRO  program  has  established  2  sets  of  criteria: 

1.  Those  criteria  established  externally  by  consultants  and 
updated  by  staff  in  order  to  study  the  system  of  care  and  track 
a  patient  through  it. 

2.  Those  criteria  (performance  standards)  set  by  local  staff 
via  a  teaching  session  and  allow  judgments  of  individual 
performance.    Weights  are  used  to  determine  the  importance  of 
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certain  risk  factors  because  some  criteria  are  more  important 
than  others  in  specific  risk  categories. 

IHS  EMCRO  documentation  noted  four  steps  which  facilitate  efforts 
to  producing  criteria: 

1.  The  data  recording  instrument  can  be  used  to  help  organize 
the  planning  process  and  to  insure  the  final  criteria  list  is 
complete. 

2.  Before  any  criteria  are  listed  on  the  form,  it  may  be  useful  to 
review  applicability.    Applicability  is  a  function  of  the 
expected  quality,  scientific  validity,  and  potential  constructive- 
ness  of  the  data  gathered  by  proposed  criteria. 

3.  Professional  guidance  from  consultants  or  local  experts  may  be 
employed  to  help  develop  initial  criteria  as  well  as  a  litera- 
ture review. 

4.  Before  the  evaluation  is  started,  local  providers  can  be 
encouraged  to  assess  and  modify  proposed  criteria  lists. 

Audits  are  performed  every  six  months  to  establish  unbiased  patterns 
of  care.    In  this  way,  health  planners  can  look  at  data  over  specific  time 
spans  and  make  conclusions  about  the  delivery  and  resources  used  for 
particular  services. 

IHS  EMCRO  documentation  indicated  two  types  of  support  are  required 
to  produce  optimal  results  from  the  FOCUS  approach,  (a)  an  effective  informa- 
tion system,  and  (b)  the  capability  to  reallocate  medical  resources  to 
solve  specific  problems. 

The  health  information  system  (HIS)  has  been  collecting  health  data 
on  patients  receiving  care  at  the  Sell's  Service  Unit  since  1969.  Informa- 
tion for  the  system  is  collected  from  a  number  of  settings  and  uses  abstracted 
data  from  encounter  forms  completed  during  field  visits,  outpatient  visits, 
inpatient  admissions,  and  inpatient  discharges.    This  information  is  used 
to  produce  patient  health  summaries  and  utilization  reports.  Ambulatory 
review  systems  being  data  dependent,  IHS-EMCRO's  approach  assumes 
quality  of  care  can  be  increased  by  improving  the  collection,  synthesis, 
and  dissemination  of  critical  health  data.    This  population-based  health 
information  system  has  been  modified  to  conduct  continuous  monitoring  of 
the  quality  through  the  introduction  of  specific  audit  programs.  Based 
on  this  goal,  a  physician  self-assessment  was  programmed  into  HIS.  The 
three  categories  of  assessment  reports  are  as  follows:    1)  population-based 
indicator  results  for  system  managers,  2)  provider  performance  reports 
for  health  worker  self-assessment,  3)  provider  performance  reports  for 
supervision  of  providers.    Population-based  indicator  results  allow  for 
checks  on  the  impact  of  the  health  system  on  the  community.    The  results 
indicate  patterns  of  care  and  demonstrate  errors  in  the  system  with  regard 
to  time  frames  for  one  patient  or  a  group  of  patients  receiving  care. 
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Provider  performance  reports  for  self-assessment  provide  individuals 
with  an  actual  list  of  their  cases,  the  number  of  times  criteria  were  met, 
the  number  not  met,  and  the  number  that  fell  above  the  standard.  The 
providers  are  also  able  to  compare  their  levels  with  average  results  of 
other  providers. 

Provider  performance  reports  for  supervision  of  providers  list 
aggregate  scores  for  all  physicians'  delivery  of  services  for  a  one-year 
period. 

A  number  of  tables  are  available  in  each  report.    The  user  can  retrieve 
desired  reports  either  by  entering  specific  access  numbers  or  by  responding 
to  a  series  of  computer  generated  options.    These  options  enable  the  user 
to  rapidly  screen  large  amounts  of  data  and  focus  on  problem  points.  The 
user  has  the  option  of  calling  for  specific  indicator  results,  aggregates 
or  various  combinations  of  aggregates.    The  user  also  has  the  option  of 
filtering  out  nonexceptional  results  based  on  miniscule  data  samples. 

In  summary,  the  importance  of  the  IHS  EMCRO  methodology  evolves  from 
its  use  of  methods  which  identify  high  risk  patients  and  high  priority 
health  problems,  and  allocate  limited  health  resources  to  identified 
problem  areas. 
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SECTION  III 

DELIVERY       SYSTEM  CHARACTERISTICS 


The  objective  of  IHS-EMCRO  is  to  improve  effectiveness,  efficiency 
and  acceptability  of  the  health  care  delivery  system  of  the  American 
Indians  by: 

1.  Developing  human  resources  for  the  operational  and  Indian 
management  of  Indian  Health  Care  Systems 

2.  Designing  the  evaluating  alternative  delivery  systems  to  respond 
to  Indian  needs 

3.  Demonstrating  the  application  and  feasibility  of  alternative 
delivery  systems  throughout  the  service 


FUNDING 

Proposed  FY  1976  funding  for  ORD  and  EMCRO  amounted  to  $291,602.  These 

figures  were  taken  from  Office  of  Planning  and  Coordination  (IHS-Tucson) 
statistics.    Allocation  was  as  follows: 


Automated  Planning/Evaluation  $83,147 

Hypertension  11,429 

Maternal  &  Child  Health  80,915 

Alcoholism  and  Drug  Abuse  25,717 

Technical  Assistance  23,071 

ORD-Duke  University  (technical  assistance)  19,606 

All  others*  48,117 

TOTAL  $291 ,602 


^Includes  administrative  staff  meeting,  orientation,  leave,  etc. 


SERVICES 

All  types  of  ambulatory  services  for  acute  and  chronic  diseases  are 
available  to  the  entire  patient  population. 


STAFF 

As  previously  mentioned,  there  are  six  practicing  physicians  and  three 
physicians'  assistants  who  rotate  through  the  various  units.    There  are  also 
two  dentists,  six  public  health  nurses  and  24  Community  Health  Representa- 
tives.   There  are  an  equal  number  of  tribal  health  groups  which  consist  of 
alcoholism  workers,  mental  health  workers,  nutrition  workers  and  wise  ones 
(senior  citizens).    There  are  also  several  social  workers,  podiatrists  and 
sanitarians.    There  are  three-four  women  who  code  and/or  abstract  informa- 
tion from  the  encounter  form  and  three-four  personnel  who  keypunch  this 
information. 
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PATIENT  POPULATION 


The  Tucson  Indian  Health  Service  is  responsible  for  the  American 
Indian  population  both  on  and  off  the  reservation  in  Southern  Arizona 
(about  17,000)  and  approximately  13,000  Alaskan  Natives.    Within  one 
year,  approximately  70%  of  the  Indian  IHS  population  will  be  seen  either 
by  a  primary  physician  or  Community  Health  Representative  and  in  five 
years  approximately  95%  will  be  seen  by  a  health  care  worker. 

According  to  the  December,  1975  statistics  (for  the  time  period  July  1, 
1974  -  June  30,  1975)  from  the  Office  of  Planning  and  Coordination  of 
0RD-IHS,  the  highest  number  of  outpatient  visits  (22.7%)  was  accounted  for 
by  visits  for  well  child  care,  surgical  follow-up,  physical  exams,  lab 
tests,  socioeconomic  problems,  family  planning  and  immunization.    The  second 
largest  category  was  Diseases  of  the  Respiratory  System  (12.5%)  followed 
by  Accidents,  Poisonings  and  Violence  (7.8%),  Infective  and  Parasitic 
Diseases  (7.7%)  and  Diseases  of  the  Skin  and  Subcutaneous  Tissue  (7.2%). 


COMPUTER  SYSTEM 

The  IHS  has  made  it  possible  for  remote  teletypewriter  terminals, 
both  fixed  and  portable,  to  provide  immediate  retrieval  of  key  health 
information  at  any  point  in  the  patient  encounter.    In  addition  to  the 
previously  mentioned  reports,  IHS  provides  the  following: 

1 •     School  Rosters  for  Strep  Throat  Screening  for  the  Streptoccol 
Disease  and  Rheumatic  Fever  Program 

2.  General  Purpose  Laboratory  Retrieval  Report  used  for  above 
program  to  tabulate  the  number  of  strep  cultures  given  for  any 
given  time  period  by  results  of  the  Beta  and  Group  A  Lab 

3.  Gastroentritis  Follow-up  Report  which  analyzes  each  child  in  the 
data  base  to  compute  a  risk  factor  based  on  various  cultures, 
and  in  accordance  with  the  assessment,  monitors  whether  appro- 
priate GE  tasks  have  been  performed 

4.  Sanitarian  Weekly  Communicable  Disease  Report  which  tabulates 
the  incidence  of  infectious  and  parasitic  disease  by  community, 
and  identifies  "repeaters"  over  a  six-month  period 

5.  Hypertension  Evaluation  Program 

6.  Hypertension  Drug  Provision  Rate 

7.  TB  Monitoring  Program  which  provides  monthly  listings  and 
monitors  most  recent  activities  relating  to  TB 

8-     Analysis  of  PPD  Skin  Tests 
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9.  The  Problem  Oriented  TB  Summary 

10.  Lists  of  individual  newborns  in  lower  weight  percentile 

11.  Prenatal  and  Newborn  Report  Lists 

12.  First  Visit  to  Newborn  Report  Lists 

13.  Cause  of  Injury  Report 

14-  Facility  Utilization  Reports 

15.  Drug  Sensitivity  Report 

16.  Population  Summary 

17-  Inpatient  Hospitalization  Statistics 

18.  Hospital  Discharge  Analysis 

19-  Contract  Health  Care  Reports 

20.  Tabulation  of  OPD  Visits  by  Clinic  Type 

21 .  Communicable  Disease  Report 

22.  Incidence  of  Disease  by  Age  Group 
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SECTION  IV 

HCMS       STUDY  INFORMATION 


IHS-EMCRO  agreed  to  participate  in  the  study  via  a  telephone  conversa- 
tion on  November  13,  1975.    Initial  documentation  was  received  on  November 
19,  1975,  which  described  the  quality  assessment  activities  of  IHS.  The 
inclusion  of  IHS  in  the  study  group  was  determined  after  suggestions  were 
received  from  various  persons  in  the  quality  assessment  field  who  recom- 
mended a  review  of  the  program. 

The  documentation  received  and  compiled  prior  to  the  site  visit  on 
February  9,  1976,  is  as  follows: 

1.  Selected  Statistics  for  the  Sells  Service  Unit  -  December,  1975 

2.  Booklet  entitled  "Quality  Appraisal  of  Ambulatory  Patient  Care: 
An  Eclectic  Approach  -  June,  1975 

3.  Booklet  entitled,  "First  Quarterly  Report  -  July  1_,  1975  - 
September  30,  1975 

4.  Summary  Description  of  the  Health  Information  System  (HIS)  - 
November,  1975 

5.  Draft  article  entitled  "A  Computer  System  for  Assessing  the 
Quality  of  Ambulatory  Care  -  March  15,  19757r" 

6.  ORD  -  IHS  Revised  Proposed  Budget  for  the  Fiscal  Year  1975  - 
July  15,  1974 

7.  Sample  Encounter  Forms  which  are  used  as  the  data  base  for  HIS. 

A  significant  amount  of  information  (140  minutes  of  taped  interviews) 
was  collected  and  questions  were  administered  to  the  Project  Coordinator 
for  Research  and  Development,  and  the  Health  Services  Research  Officer. 

The  people  interviewed  had  been  involved  with  the  QAP  since  its 
inception  and  were  able  to  provide  an  excellent  overview  of  quality  review 
issues  and  program  content.    An  IHS-EMCRO  programmer  demonstrated  the  use 
of  the  HIS  and  Medical  Care  Evaluation  Project  (MCEP),  as  well  as  fielding 
study  staff  questions. 

The  following  topics  were  covered  during  the  interview: 

1.  Development  and  implementation  of  MCEP 

2.  Funding  for  MCEP  activities,  including  specific  quality  assess- 
ment activity  funding  and  computer  (HIS)  funding 

3.  Staff  resources  involved  in  MCEP 
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4.  Unique  characteristics  of  Indian  Health  Services,  including 
population  characteristics  and  HIS 

5.  Problems  encountered  in  implementing  the  use  of  HIS  with  regard 
to  quality  assessment  activities 

6.  Components  of  the  review  process,  including  identification  of 
problems,  criteria  development,  computer  and  manual  review,  and 
audit. 

7.  Use  of  encounter  form  in  MCEP  and  reliability  of  computing  forms 
into  HIS 

8.  MCEP's  present  level  of  operation 

9.  The  cost  of  developing  and  implementing  MCEP 

10.  Medical  service  risk  models 

11.  Type  and  amount  of  "spinoff"  studies  at  IHS 

12.  Attitudes  toward  process  and  outcome  issues 

13.  Future  activities  in  quality  assessment  involving  HIS  and  MCEP 

In  addition,  MCEP  computer  demonstrations  were  provided  by  a  member 
of  IHS  staff. 
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NORTH  COMMUNITIES  HEALTH  PLAN,  INC. 
Evanston,  Illinois 


SECTION  I 
INTRODUCTION 


North  Communities  Health  Plan,  Inc.,  (NCHP)  is  a  free-standing  prepaid 
medical  care  system,  delivering  services  to  an  enrolled  population  of  7,000 
in  a  service  area  which  includes  the  far-north  side  of  Chicago  and  thirteen 
north  suburbs.    NCHP  is  a  qualified  health  maintenance  organization  (HMO) 
which  contracts  with  a  physician  service  corporation  to  provide  professional 
services  to  beneficiaries. 

NCHP  has  been  in  operation  since  May  1,  1975,  and  was  federally  qualified 
as  a  Health  Maintenance  Organization  on  May  18,  1975.    NCHP  operates  one 
facility  staffed  by  sixteen  physicians  who  are  salaried  members  of  North 
Communities  Health  Plan  Physicians  Service  Corporation.    There  are  six  internists, 
(three  full-time),  two  pediatricians  (one  full-time),  two  family  practitioners, 
three  obstetrician-gynecologists,  and  three  part-time  psychiatrists.  Secondary- 
care  physician  services  (surgery,  orthopedic  surgery,  urology,  etc.)  are  provided 
on  the  basis  of  fixed-schedule  fee-for-service  contracts;  beneficiaries  see 
secondary-care  physicians  only  upon  the  referral  of  one  of  their  salaried, 
primary-care  physicians  or  upon  referral  of  one  of  the  nurse-practitioners. 
There  are  three  full-time  and  one  half-time  nurse-practitioners.    The  health 
associates  director  is  also  a  graduate  nurse-practitioner. 

Other  health  professionals  consist  of  a  full-time  psychiatric  social  worker 
who  acts  as  the  coordinator  of  the  mental  health  group,  a  part-time  psychiatric 
social  worker,  and  two  part-time  podiatrists. 

The  general  organizational  structure  includes  a  21 -member  board  of  trustees 
elected  by  the  membership,  an  executive  director,  a  medical  director,  and  a 
health  associates  director.    The  executive  director  is  responsible  for  marketing 
and  business  affairs;  the  medical  director  is  responsible  for  medical,  technical 
and  scientific  affairs;  the  health  associates  director  is  responsible  for 
administration  and  clinical  affairs  of  the  health  center  and  the  nurse- 
practitioners  program. 
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SECTION  II 

QUALITY     ASSURANCE  PROGRAM 


BACKGROUND  AND  FORMALLY  PLANNED  ACTIVITIES 

Certain  aspects  of  the  NCHP  quality  assurance  program  have  been  in 
effect  since  NCHP  opened,  but  the  full  program  has  been  operational  only 
since  January,  1976, and  it  is  not  specifically  budgeted.    Both  explicit 
and  implicit  standards  of  care  are  administered  by  the  Quality  Control 
Committee  (QCC).    The  problem-oriented  medical  record  (POMR)  is  the  primary 
data  source  for  process  and  outcome  audits  using  explicit  criteria,  utili- 
zation review,  and  membership  satisfaction  review.    An  automated  information 
retrieval  system  is  used  for  all  aspects  of  management  review. 

The  general  outline  of  the  NCHP  Quality  Assurance  Program  (QAP)  obli- 
gates contracted  physicians  to  develop  and  implement  certain  standards  of 
medical  care  for  the  entire  beneficiary  population.    These  explicit  and 
implicit  standards  are  administered  by  the  QCC  which  is  co-chaired  by  the 
medical  director  and  the  health  associates  director.    The  medical  record  is 
the  data  source  for  review  of  physician  and  nurse  practitioner  care,  while 
the  automated  management  information  system  produces  data  for  analysis  of 
hospital  and  ambulatory  utilization. 

A  quality  assurance  program  was  developed  at  NCHP  for  three  reasons: 

1.  To  meet  federal  regulations  of  the  HMO  Act  of  1973,  Section 
108.110  (J)  (1)  through  (4)  and  (K)  through  (R) 

2.  The  interest  of  the  medical  director  in  a  formal  quality 
assurance  program 

3.  The  requirements  of  a  federally  supported  delivery  system 
like  NCHP  to  fully  document  all  activities 

NCHP  emphasizes  recruitment  screening  as  an  important  quality  assurance 
activity.    Because  of  the  academic  setting  of  the  Evanston  medical  community, 
recruited  physicians  generally  have  established  reputations  within  the  community 
and  are  associated  with  educational  activities  in  medicine. 

Documents  from  NCHP  describe  various  plans  by  which  quality  assurance 
activities  have  been  implemented  and  the  organizational  mechanisms  developed 
for  conducting  these  activities.    NCHP  does  not  have  any  funds  earmarked 
specifically  for  quality  assurance  activities. 

NCHP  feels  a  special  responsibility  to  assure  and  document  high  standards 
of  care  for  the  enrolled  population  through  a  system  of  peer  review.  The 
foundation  of  NCHP  quality  assurance  activities  is  the  Medical  Service  Agree- 
ment, which  requires  contracted  physicians  to  develop  and  implement  certain 
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standards  of  medical  practice  and  to  establish  specific  indicators  for  review 
of  care  based  on  those  standards. 

The  documentation  cites  several  quality  control  mechanisms: 

1.  Problem  Oriented  Medical  Record,  (POMR)  which  facilitates 
quality  review  because  it  documents  medical  processes  more 
precisely  than  other  types  of  records 

2.  Use  of  encounter  forms  to  trace  utilization 

3.  Use  of  information  retrieval  systems  to  document  all  facets 
of  the  NCHP  delivery  system 

4.  Professional  Quality  Control  Committee,  which  will  conduct 
reviews  of  randomly  selected  outpatient  medical  records 

5.  Membership  Satisfaction  Subcommittee,  which  will  review 
members'  grievances,  relate  these  problems  to  the  performance 
of  the  health  plan,  and  act  as  the  conduit  between  the  NCHP 
personnel  and  the  enrolled  population 

6.  Examination  of  various  process  criteria  (e.g.,  enrollee  satis- 
faction, patient  satisfaction,  health  professional  satisfaction, 
immunization  rates,  utilization  rates)  and  outcome  criteria 
(e.g.,  mortality  rates,  infant  mortality  rates,  duration  of 
disabil ity) 

The  specifics  of  the  review  procedure  are: 

1.  Inpatient  and  outpatient  chart  reviews;  choosing  a  certain  percentage 
of  randomly  selected  charts 

2.  Specific  case  review  and  discussion  of  complex  or  problem 
cases 

3.  Patient  interviews 

4.  Death  conferences,  structured  along  the  lines  of  a  "clinical - 
pathological  conference" 

As  an  integral  part  of  the  quality  assurance  program,  the  documentation 
notes  that  "systematic  data  collection"  will  be  implemented.    The  sources 
for  "systematic  data  collection"  are  the  encounter  form,  the  POMR  and  the 
management  system.    Data  collection  is  the  responsibility  of  the  director 
of  health  information,  who  manages  medical  record  services. 

The  medical  director  is  given  a  central  role  in  the  NCHP  quality 
assurance  program.    As  cochairman  of  the  QCC,  he  is  responsible  for  overseeing 
review  functions  and  taking  action  on  recommendations  from  the  QCC  to  insure 
that  "changes  needed... to  guarantee  quality  of  care  will  be  instituted." 

Quality  assurance  is  approached  indirectly  in  several  ways.  All 
salaried  physicians  who  are  members  of  the  Service  Corporation  are  required 
to  devote  one  week  per  year  to  attending  a  medical  education  meeting  or 
course.    Health  care  provider  (physicians,  nurse-practioners ,  podiatrists, 
psychiatric  social  workers)  meetings  allow  professional  staff  to  review  the 
entire  NCHP  operation.    Health  center  staff  meetings,  chaired  by  the 
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health  associates  director,  review  all  operational  activities  of  the  health 
center.    The  development  of  a  Drug  Use  Profile  is  noted  as  a  quality  assurance 
mechanism  to  evaluate  drug  usage  of  the  beneficiary  population. 

NCHP  documentation  generally  covers  all  facets  of  the  prospective  and 
existing  quality  review  programs.    Documentation  did  not  address  the  pro- 
cedural aspects  of  standards  development,  specific  feedback  guidelines  or 
methods  to  measure  the  impact  of  the  quality  assurance  program. 


OPERATIONAL  ACTIVITY 

The  operational  status  of  NCHP ' s  ambulatory  quality  assurance  program 
was  determined  during  a  site  visit  conducted  April  29,  1976,  and  via  staff 
review  of  the  Quality  Control  Committee's  (QCC)  minutes.  Operational 
activities  fall  into  five  broad  categories: 

1.  Establishment  of  quality  assurance  structure  and  assignment  of 
responsibil ities 

2.  Conducting  audits  and  reviewing  the  audits 

3.  Initiation  of  review  guidelines 

4.  Quality  assessment  activities  through  the  Professional  Quality 
Control  Committee 

5.  External  review  of  NCHP  conducted  by  independent  research  firm 

The  NCHP  personnel  made  two  general  points  regarding  physicians  hired 
at  NCHP,  the  environment  of  the  medical  community,  and  the  importance  of 
each  of  those  areas  in  the  development  and  implementation  of  formal  quality 
assurance  activities: 

1 .  Attitudes  of  the  medical  director  and  health  associates 
director  toward  the  types  of  providers  recruited.  Personal 
acquaintance  with,  and  careful  selection  of  recruited  providers 
(or  nurse  practitioners),  "known"  to  be  very  good  physicians 
allows  less  emphasis  on  formal  quality  assurance  procedures 
during  the  first  year  of  operation.    Furthermore,  Evanston 
Hospital,  where  most  recruited  physicians  practice,  has  a  solid 
reputation  for  high-quality  medicine.    Recruiters  have  extensive 
knowledge  of  practitioners  in  the  community. 

2.  Establishing  NCHP  took  over  four  years.    The  medical  director 
argued  for,  and  finally  won,  support  from  many  physicians 
predominantly  practicing  in  a  fee-for-service  mode.    The  strategy 
was  to  recruit  community  physicians  with  good  reputations  to  lend 
credibility  to  the  program  from  the  beginning.    Opposition  to  a 
prepaid  practice  also  required  NCHP  to  document  the  type  of  care 
delivered.    The  recruitment  aspects  of  quality  assurance  have  been 
of  a  higher  priority  than  initiation  of  formal  quality  assurance 
methods. 
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In  terms  of  structure,  NCHP  has  established  the  QCC,  which  includes  the 
following  members:    medical  director,  health  associates  director,  one 
primary-care  physician,  the  chairman  of  the  Department  of  Medicine  of 
Evanston  Hospital,  and  health  information  director. 

QCC  conducted  meetings  in  March  and  April,  1976,  attended  by  all 
committee  members,  and  cochaired  by  the  medical  director  and  the  health 
associates  director.    During  these  initial  meetings  a  sample  audit  of  nine 
medical  records  was  conducted  and  discussed.    Prior  to  the  first  meeting 
a  review  audit  sheet  was  developed  as  a  guideline.    This  reporting  form 
asks  twelve  questions  dealing  with  these  data  elements: 


1. 

Medical  record  legibility 

2. 

Diagnosis  and  treatment  rationality,  relevancy,  and  adequacy 

3. 

Appropriateness  of  laboratory  tests 

4. 

Appropriate  physician  utilization 

5. 

Appropriate  follow-up 

6. 

Action  taken  regarding  abnormal  laboratory  and  X-ray  results 

7. 

Date  of  review 

8. 

Medical  record  number 

9. 

Auditor 

10. 

A  list  of  diagnoses  or  problems 

11. 

A  list  of  medications 

12. 

Disposition  of  the  case 

The  reviewer  must  indicate  "yes"  or  "no"  answers  along  with  an  explanation. 
The  QCC  conducted  the  initial  audit  by  evaluating  these  data  elements  in 
the  selected  medical  records.    Since  then,  general  guidelines  for  use  of 
the  audit  sheet  have  been  developed  by  the  health  information  director  for 
the  entire  provider  staff. 

Completed  audits  are  administered  by  the  health  information  director 
as  follows: 

1.  Audit  topic  is  approved  by  the  QCC 

2.  Medical  records  are  randomly  selected  based  on  some  indicator 
(e.g.,  the  March  audit  indicator  was  complete  health  assessments) 

3.  Medical  records  are  distributed  to  all  primary  care  physicians 
and  nurse-practitioners  for  actual  review 

4.  Results  are  tabulated  by  the  health  information  director;  they 
are  discussed  at  the  QCC  meetings  and  actions  are  recommended; 
results  are  distributed  to  all  providers 

Information  concerning  review  activities  of  individual  providers  is  also 
maintained  by  the  health  information  director  and  reviewed  by  the  medical 
director  and  the  QCC. 
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The  medical  director  indicated  no  action  was  taken  based  on  the  results 
of  the  first  audit;  however,  results  of  the  audit  were  distributed  to  the 
primary-care  physician  staff  and  the  nurse-practitioners. 

The  implementation  of  the  entire  quality  assurance  program  as  outlined 
in  documentation  is  still  under  discussion  within  the  QCC  and  Health  Care 
Provider  meetings. 

Some  informal  quality  assurance  activities  were  being  conducted  within 
the  context  of  the  Health  Care  Provider  meetings,  which  began  May,  1975. 
These  meetings  are  held  monthly  and  all  clinical  providers  are  required  to 
attend.    Review  of  the  minutes  for  all  Health  Care  Provider  meetings 
revealed  the  following: 


1.  The  meetings  are  well  attended  and  last  approximately  2  hours 

2.  Hospital  utilization,  hospital  length  of  stay  and  physician 
referrals  comprise  the  major  business  of  the  meetings 

3.  There  is  little  discussion  of  ambulatory-primary  quality  of 
care  issues 

4.  Health  center  administrative  issues  (e.g.,  appointment-making 
procedures,  medical  records,  laboratory  services)  are  discussed 
often 


The  final  area  of  quality  assurance  activity  cited  as  operational 
deals  with  an  external  review  completed  by  INTERSTUDY,  Minneapolis,  Minnesota.  Al- 
though this  activity  is  not  part  of  the  QCC's  program,  the  medical  director 
indicated  the  study  gave  NCHP  a  good  rating  and  made  several  suggestions  about 
the  delivery  of  services  at  NCHP.    The  three-day  INTERSTUDY  evaluation  reviewed 
150  medical  records  based  on  a  systematic  audit  criterion  and  procedure.  The 
results  were  summarized  and  sent  to  NCHP  and  discussed  at  a  Health  Care  Provider 
meeting. 


ADDITIONAL  COMMENTS 

NCHP  personnel  indicated  several  unique  characteristics  of  the  Plan  have 
affected  and  will  continue  to  affect  the  development  and  implementation  of 
quality  assurance  activities: 

1.  The  long  historical  development  of  NCHP,  evolving  from  a 
heavy  consumer  orientation  to  a  concern  for  the  busines 
survival  of  the  program 

2.  A  close  association  with  a  major  medical  teaching  facility 

3.  A  costly  open-enrollment  experience  which  threatened  the  fiscal 
soundness  of  NCHP 


! 
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4.    Being  a  federally  qualified  HMO 

The  major  problem  in  systematic  implementation  of  NCHP's  quality 
assurance  program  is  the  emphasis  of  administrative  and  clinical  staff  on 
assuring  NCHP's  survival  as  a  health  plan  rather  than  on  quality  assessment 
activities  during  this  early  stage  of  NCHP's  development.    This  is  made 
evident  by  examining  the  type  of  quality  assurance  issues  addressed  by  NCHP 
during  the  early  operational  period.    The  aspects  of  the  quality  assurance 
program  addressed  since  opening  day  were  secondary  care  referral  cases  and 
review  of  utilization  data  (both  outpatient  and  inpatient).  Approximately 
six  months  from  initial  delivery  of  services,  NCHP  began  to  address  more 
substantive  quality  issues  through  review  of  medical  records. 
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SECTION  III 

DELIVERY     SYSTEM  CHARACTERISTICS 


North  Communities  Health  Plan,  Inc.  evolved  from  a  consumer-initiated 
program  (Evanston  Medical  Consumers,  Inc.)  to  official  incorporation  in 
1970-71.    The  Physicians'  Service  Corporation  was  formed  in  1974  to  facili- 
tate recruitment  of  physicians,  to  provide  physicians  with  a  formal  corporate 
structure  to  work  from  and  to  take  advantage  of  certain  IRS  rulings  for 
setting  up  retirement  programs. 

Evanston  Medical  Consumers,  Inc.  (EMC)  received  initial  funding  from 
the  Illinois  Regional  Medical  Program  to  begin  developmental  activities. 
The  Robert  Wood  Johnson  Foundation  followed  with  an  $88,000  grant  for 
educational  and  developmental  activities,  and  another  $100,000  if  EMC-NCHP 
could  match  that  sum,  which  they  did  shortly  before  NCHP  became  operational. 
The  Chicago  Community  Trust  Foundation  awarded  NCHP  $45,000  to  develop  a 
nurse-practitioner  program  for  the  health  plan.    NCHP  received  federal  HMO 
qualification  in  May  1975,  with  developmental  funding  of  $247,000. 

Operational  funding  is  derived  from  two  sources: 

1.  Prepaid  enrollee  dollars  from  individuals  and  employer- 
employee  groups 

2.  Financial  support  (credit  line)  to  cover  initial  operating 
deficits,  from  Illinois  Hospital  and  Health  Services,  Inc. 
(Rockford  Blue  Cross) 

The  organizational  structure  consists  of  several  units:    the  Board  of 
Trustees,  the  executive  director,  the  medical  director,  and  the  health 
associates  director.    The  Board  of  Trustees  must  consist  of  50%  consumer 
membership  and  includes  several  subcommittees:    Professional  Affairs,  Finance 
and  Budget,  Membership  Satisfaction,  Personnel,  Program  and  Benefits  and 
Executive.    The  executive  director  administers  the  marketing  and  business 
operations  with  a  staff  of  six  in  marketing  and  enrollment  and  four  in  the 
business  office.    The  medical  director  is  responsible  for  the  medical, 
scientific  and  technical  aspects  of  NCHP,  and  supervises  the  following 
distribution  of  primary-care  physicians,  who  are  salaried  members  of  the 
Physicians'  Service  Corporation: 

1 .  Internal  Medicine 

2  full -time  physicians 
5  part-time  physicians 

2.  Pediatrics 

1  full-time 

1  part-time  (37%) 
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3.  Family  Practice 

2  part-time  (50%  and  40%  respectively) 

4.  Obstetrics  and  Gynecology 

3  part-time  (20%  time  each) 

5.  Psychiatric 

3  part-time  (20%,  15%,  11%  time  respectively)  • 

The  medical  director  also  supervises  secondary-care  physicians  to  whom 
NCHP  patients  are  referred  for  consultations  and  specialty  care.  Fifty 
secondary-care  physicians  have  signed  agreements  with  the  Physicians'  Service 
Corporation.    The  director  of  laboratory  service  also  reports  directly  to  the 
medical  director  regarding  medical -technical  problems.    Two  full-time  technicians 
and  one  half-time  technician  comprise  the  laboratory  force. 

The  health  associates  director  is  responsible  for  administration  of  the 
health  center  and  all  employees.    The  three  full-time  and  one  half-time 
nurse-practitioners  report  directly  to  her,  along  with  two  clinical  assistants 
and  the  health  information  director.    The  health  information  director  super- 
vises medical  records,  switchboard,  and  transcription  and  reception,  a  total 
of  8.7  full-time  and  four  part-time  employees. 

The  health  associates  director  is  also  responsible  for  the  administrative 
supervision  of  the  mental  health  unit,  staffed  by  a  full  time  psychiatric 
social  worker  as  coordinator,  a  part-time  psychiatric  social  worker,  and  three 
part-time  psychiatrists. 

The  medical  director  is  responsible  for  all  the  medical  matters  and 
policies  in  the  health  center. 

The  executive  director,  medical  director,  and  health  associates  director 
report  directly  to  the  board  of  trustees. 

Nurse-practitioners  participate  in  delivery  of  basic  ambulatory  services, 
and  often  perform  initial  health  evaluation  of  new  members  in  conjunction  with 
a  physician.    Rockford  Blue  Cross  underwrites  out-of-area  emergency  care 
services,  reinsures  NCHP  for  losses  over  $5,000  per  beneficiary  in  one  year, 
and  does  claims  administration.    Prescription  drugs  are  available  to  NCHP 
members  at  competitive,  free-standing  pharmacies.    The  NCHP  benefit  package 
is  comprehensive  and  covers  all  outpatient  medical  and  health  services,  inclu- 
ding laboratory  tests,  X-rays,  injections,  immunizations  and  mental  health 
services.    All  hospital  services  are  covered.    All  services  are  unlimited, 
except  for  mental  health  outpatient  and  inpatient  services. 

The  medical  record  employed  by  NCHP  is  a  Problem  Oriented  Medical  Record 
(P0MR)  which  includes  a  problem  list,  medication  list,  immunization  and 
screening  record  and  laboratory  reports  on  the  left  side  of  the  chart.    On  the 
right  side  of  the  chart  there  are  other  sheets  for  the  adult  medical  history 
and  systemic  review,  or  for  the  pediatric  history  with  immunization  history 
and  physical  exam.    The  remaining  elements  on  the  right  side  of  the  chart  are 
divided  into  four  sections:    special  sheets,  X-ray,  EKG,  consultation  and 
progress  notes  continuation  sheets.    Encounter  forms  are  filled  out  for  every 
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patient  encounter  and  are  the  data  source  for  t her  automated  information 
system.    The  encounter  form  includes  the  following  data  elements:  provider 
identification,  patient  identification,  date  of  service,  purpose  of  visit 
(10  coded  subelements) ,  services  (20  coded  subelements) ,  X-rays  ordered 
(14  coded  subelements),  future  orders  (for  laboratory  workups)  and  immuni- 
zations.   The  reverse  side  of  the  encounter  form  lists  86  possible  laboratory 
procedures  and  their  codes.    Automated  information  retrieval  services  are 
available  from  Rockford  Blue  Cross  which  supplies  claims  administration 
and  utilization  reports. 
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SECTION  IV 

HCMS     STUDY  INFORMATION 


NCHP  agreed  to  participate  in  the  study  via  correspondence  dated 
November  19,  1975.    The  following  documentation  was  received  and  compiled 
before  and  after  the  site  visit: 

1.  Copy  of  medical  record 

2.  Narrative  Statement  Outlining  Historical  development  of 
NCHP  (July,  1975) 

3.  Quality  Assurance  Committee  minutes  (March  31,  1976;  April  21,  1976) 

4.  Health  Care  Providers  Meeting  agenda  and  minutes  (November  7,  1975; 
December  11,  1975;  January  22,  1976;  February  13,  1976;  March  11,  1976) 

5.  Policy  Statement:    Confidentiality  of  NCHP  members  medical  records 

6.  Results  of  February  audits  (March  31,  1976) 

7.  Hospital  Census  Data  (January-March,  1976) 

8.  Physician  Compensation  Schedule  (February  8,  1975) 

9.  Copy  of  encounter  form  (November,  1975) 

10.  Audit-Reporting  form  (November,  1975) 

11.  Employment  Agreement  for  NCHP  Physicians  (full-time) 

12.  Employment  Agreement  for  NCHP  Physicians  (part-time) 

13.  Medical  Service  Agreement,  between  NCHP,  Inc.,  and  NCHP 
Physicians  Service  Corporation  (April  23,  1975) 

14.  NCHP,  Inc.,  By-Laws  (June,  1975) 

15.  Results  Patient  Questionnaire  (March  3,  1976) 

16.  Laboratory  and  Radiology  Utilization  by  Provider  (February,  1976) 

17.  Subscriber  Benefits  (May,  1975) 

18.  NCHP,  Inc.  (HMO  Qualification  Document,  1975) 

19.  NCHP,  Inc.  Quality  Assurance  Program  document 

This  documentation  outlined  every  aspect  of  development  and  implementa- 
tion of  the  delivery  system.    Information  from  this  documentation  is  compiled 
along  with  information  collected  during  site  visit  interviews. 

HCMS  staff  site  investigators  visited  NCHP  on  April  29,  1976.  The 
following  personnel  were  interviewed: 

Medical  Director 
Health  Associates  Director 
Health  Information  Director 
Executive  Director 

Total  interview  time  amounted  to  approximately  six  and  one-half  hours. 
The  interviews  were  not  tape  recorded,  but  the  two  site  investigators  took 
notes  during  each  interview.    Interviews  covered  several  topics  including: 
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1.  Historical  development  of  NCHP,  covering  period  from  1971 
through  1976 

2.  Initiation  of  prepaid  practice  in  Evanston,  where  fee-for- 
service  is  predominant  mode 

3.  Recruitment  of  physicians  and  their  role  in  establishing  NCHP 

4.  Development  of  strong  nurse-practitioner  role  in  NCHP 

5.  Development  and  initiation  of  quality  assurance  program  elements 

6.  The  role  of  provider  meetings  in  developing  commitment  to  a  formal 
quality  assessment  program 

7.  Initiation  of  specific  quality  assurance  activities  in  NCHP 
administrative  structure 

8.  Management  information  system 

9.  First-year  problems  of  NCHP  in  delivery  of  services  and  remaining 
financially  viable 

10.  Marketing  strategy  at  NCHP 

11.  Future  activities  in  quality  assurance 
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SOUND  HEALTH  ASSOCIATION 


Tacoma,  Washington 


SECTION  I 
INTRODUCTION 


The  Sound  Health  Association  (SHA)  in  Tacoma,  Washington,  is  one  of 
three  qualified  health  maintenance  organizations  (HMO)  in  this  study.  SHA 
is  a  free-standing  prepaid  practice  delivering  medical  services  to  a 
patient  population  of  3,100  employing  four  FTE  physicians,  seven  auxiliary 
medical  personnel  and  22  support  staff.    A  full  range  of  medical  services, 
including  physician  office  visits,  hospital  coverage,  extended  care  coverage 
and  out-of-area  coverage  is  provided. 

The  organizational  structure  consists  of  the  Board  of  Directors,  Execu- 
tive Director,  Medical  Director,  and  Committees  of  the  Board  of  Directors  and 
Health  Center  Staff. 

The  quality  assurance  program  activities  at  SHA  are  largely  informal, 
although  formal  program  elements  have  been  outlined  generally.    SHA  personnel 
indicated  their  ability  to  document  and  administer  a  formal  quality  assurance 
program  is  limited  since  the  organization  is  a  new  one  which  must  emphasize 
financial  solvency.    SHA  was  included  in  this  study  because  of  their  status 
as  a  federally  qualified  health  maintenance  organization,  which  requires  a 
formal  quality  assurance  effort. 
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SECTION  II 


QUALITY     REVIEW  PROGRAM 


SHA  documents  describe  the  quality  assurance  program  as  one  that  examines 
the  relationship  between  diagnostic  (process)  procedures,  treatment  regimens 
and  the  medical  results  of  that  treatment.    SHA  assumes  responsibility  of  care 
for  a  specific  population  "and  therefore  is  concerned  about  the  whole  chain  of 
events  that  leads  one  of  its  members  from  a  problem  .  .  .  through  an  appropriate 
outcome."    Documents  provided  outlined  a  quality  assurance  program  on  a  general 
level,  describing  some  specific  program  elements  and  objectives.    As  a  federally 
qualified  HMO,  SHA  is  required  to  comply  with  regulations  (Section  110.108  (j) 
(1)  through  (4)  and  (k)  through  (r))  outlining  an  HMO's  program  emphasis  for  an 
ongoing  quality  assurance  program. 

Areas  of  quality  assessment  responsibilities  are  outlined  for  SHA  profes- 
sional and  administrative  employees,  as  well  as  outside  professional  and  govern- 
mental entities.    Each  group  of  health  care  providers  (e.g.  Board  of  Directors, 
physicians,  nurses,  etc.)  must  adopt  regular  and  active  review  procedures  de- 
signed to  address  three  areas: 

1.  Evaluation  of  staff  by  self  and  by  people  served 

2.  Evaluation  of  providers  by  peers 

3.  Evaluation  of  any  supervising  staff  member  in  all  groups 

The  "evaluation  procedures  developed  by  each  group  must  be  relevant  to  its 
responsibilities  and  acceptable  to  the  Executive  Director."    The  Executive 
Director  and  Medical  Director  are  responsible  for  integration  of  review  pro- 
cedures into  daily  operations.    Quality  evaluation  records  must  be  reviewed 
regularly  within  each  provider  group  or  at  intervals  specified  by  the  Executive 
Director,  who  must  prepare  "an  overall  quality  evaluation  report  for  the  Board 
of  Directors." 

The  final  area  of  review  activities  deals  with  an  "annually  appointed  group 
of  outside  reviewers."    Specific  expertise  required  by  the  review  team,  areas 
of  investigations,  and  data  concepts  are  outlined  in  documentation. 

SHA  documents  outlined  specific  delivery  system  components  which  must  be 
covered  in  the  quality  assurance  program.  These  are  divided  into  three  cate- 
gories:   (a)  structure,  (b)  process,  and  (c)  outcome. 

The  structure  component  would  examine  the  following  elements: 

1.  Licensing  requirements  for  personnel,  organization,  facilities 

2.  Equipment,  facilities,  etc. 

3.  Organizational  structure 

4.  Curriculum  requirements  for  obtaining  professional  position 

5.  Continuing  education 
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The  evaluation  of  components  within  the  context  of  a  formal  quality  assurance 
program  would  be  implemented  by  the  Executive  Director's  office. 

The  process  component  can  be  conducted  from  two  approaches: 

1.  According  to  actual  steps  taken  in  diagnosis  and  treatment 
of  medical  problems 

2.  According  to  a  set  of  "proper  actions"  or  standards  allowing 
judgment  of  appropriate/inappropriate  action 

The  documents  note  this  component  would  be  implemented  through  some  type 
of  utilization  review  and/or  medical  audit. 

The  outcome  component  would  examine  the  "actual  outcomes  of  medical 
encounter(s)  rather  than  the  processes"  for  the  SHA  population.    SHA  plans  to 
measure  outcome  through  examination  of  the  following  data:    (a)  infant  mortality 
rate,  (b)  surgical  complications,  and  (c)  incidence  of  certain  diseases. 


OPERATIONAL  ACTIVITIES 

Besides  one  enrollee  survey,  study  staff  could  document  only  limited 
operational  activities  that  were  associated  with  SHA's  planned  quality  assurance 
program  elements.    During  the  interview,  however,  SHA  personnel  indicated  several 
quality  assurance-type  activities  had  been  implemented.    Although  these  activi- 
ties do  not  reflect  a  systematic  formal  program,  SHA  indicated  they  were  part  of 
a  quality  assurance  mechanism. 

1.  Patient  satisfaction  questionnaire 

2.  Examination  of  SHA  referral  structure 

3.  Informal  medical  record  review 

4.  Continuing  education  for  SHA  physicians 

5.  Review  of  complaints/criticisms  from  enrol  lees 

In  addition  to  these  activities,  SHA  personnel  indicated  development  of 
program  plans  (see  preceding  section)  should  be  considered  operational. 

SHA  implemented  one  patient  questionnaire  (No.  1  above)  in  August-September, 
1975.    A  random  sample  of  100  SHA  enrollees  received  the  18- item  questionnaire; 
36  enrollees  responded.    SHA  personnel  indicated  the  survey  was  a  valuable  tool 
for  identifying  areas  of  enrollee  dissatisfaction. 

Review  of  enrollee  complaints  (see  No.  5  above)  is  closely  related  to  the 
survey.    The  Executive  Director's  office  is  responsible  for  addressing  complaints. 
SHA  personnel  indicated  one  good  barometer  for  judging  quality  of  care  is  a 
record  of  enrollee  complaints. 

The  next  three  activity  areas  were  items  noted  as  quality  assurance  activi- 
ties during  interviews  with  SHA  personnel.    The  examination  of  referral  struc- 
ture (No.  2  above)  is  a  utilization/management  review  conducted  by  the  Executive 
and  Medical  Directors.    SHA  personnel  indicated  this  activity  was  a  normal 
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administrative  activity,  and  not  considered  a  formal  quality  assurance  element. 
The  review  of  medical  records  (see  No.  3  above)  was  conducted  by  the  Medical 
Director.    The  Director  did  not  develop  an  explicit  criteria  or  decision 
guidelines  because  he  wanted  input  from  more  physicians  before  adopting 
guidelines.    He  did  review  a  number  of  records  for  medical  outcomes.  The 
Director  indicated  this  review  was  conducted  to  assess  new  physicians'  per- 
formance.   The  charts  reviewed  were  not  selected  in  a  systematic  fashion. 
The  area  of  continuing  medical  education  (see  No.  4  above)  was  considered  a 
specific  quality  assurance  item  conducted  to  insure  that  SHA  physicians  had 
the  opportunity  to  continue  learning  while  practicing.    Specific  guidelines 
(e.g.,  number  of  hours  given  for  education,  what  type  of  educational  activity 
was  required,  etc.)  from  different  specialty  boards  are  used  to  govern  these 
activities. 

In  general,  there  have  been  some  activities  oriented  toward  quality 
assurance,  although  not  in  a  formal  manner.    SHA  personnel  indicated  they 
consider  quality  assurance  "important",  but  that  it  must  be  implemented  in 
perspective  with  many  other  organizational  objectives. 


ADDITIONAL  COMMENTS 

Thro.,  ,/iout  the  interview,  several  interesting  aspects  of  SHA  delivery 
setting  were  noted,  and  described  in  relation  to  altering  potential  quality 
assurance  activities. 

The  major  factors  affecting  quality  assurance  were: 

1.  Relatively  small  number  of  encounters  for  review 

2.  Emphasis  on  financial  and  organizational  stability  of 
SHA  delivery  system 

3.  Staff  time  committed  to  essential  administrative  duties 

4.  Relatively  small  clinical  staff  making  formal  review  functions 
less  applicable  and  desirable 
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SECTION  III 


DELIVERY  SYSTEM 


Sound  Health  Association  (SHA)  divides  its  services  into  two  major 
categories;  health  maintenance  and  medical  treatment  activities. 

As  a  qualified  HMO,  SHA  offers  a  broad  range  of  benefits  to  the  enrolled 
population.    Health  maintenance  activities  include  health  education,  counseling, 
behavioral  psychological  programs,  self-care,  patient  education,  health  hazard 
appraisal,  preventive  dental  services  and  diet-nutrition  instruction.  The 
medical  treatment  program  deals  with  physician  priority  services,  emergency 
services,  inpatient  services  (contracted  providers),  extended  care  services 
and  home  health  services.    SHA  organizational  structure  includes  the  following 
administrative  personnel: 

1.  A  21 -member  Board  of  Directors,  elected  by  the  membership 
population 

2.  Executive  Director 

3.  Administrative  Assistant  to  the  Executive  Director 

4.  Medical  Director 

5.  Director  of  Finances 

6.  Director  of  Enrollment 

7.  Health  Center  Administrator 

There  are  five  standing  committees  and  one  ad  hoc  committee  emanating  from  the 
Board  of  Directors: 

1.  Finance  and  Management  Committee 

2.  Health  Benefits  Committee 

3.  Cross  Currents  Committee 

4.  Members  Relation  Committee 

5.  Joint  Conference  Committee 

6.  Building  Committee  (ad  hoc) 

The  Board  of  Directors  dictates  policy  to  the  Executive  Director  who  is 
responsible  for  the  total  program. 

SHA  serves  a  patient  population  of  3,100  enrol  lees.    At  the  time  of  the 
survey,  65  -  70%  of  the  enrollees  received  care  on  a  prepaid  basis,  10  -  15% 
fee-for-service  and  12%  Medicare/Medicaid .    In  terms  of  subscriptions, 
80%  of  the  members  are  employee-employer  group  members  and  15%  are  individual 
subscribers.    There  are  no  third-party  insurance  carriers  involved  in  the 
program,  except  through  fee-for-service  patients.    SHA  received  a  $1,000,000 
developmental  grant  from  DHEW  on  December  19,  1974. 

The  medical  record  employed  at  SHA  is  a  problem  oriented  medical  record 
(P0MR),  with  an  active  list  at  the  front  of  the  chart  which  notes  the  problem 
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code,  date  of  resolution  and  inactive  problem.    Progress  notes  are  in  chronolo- 
gical order,  using  the  subjective,  objective,  assessment,  plans  (SOAP)  format. 
The  final  section  is  for  inclusion  of  lab  and  X-ray  information,  as  well  as 
information  for  other  auxiliary  services. 
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SECTION  IV 


HCMS     STUDY  INFORMATION 


SHA  agreed  to  participate  in  the  study  in  correspondence  from  the  Execu- 
tive Director  (November  6,  1975).  On-site  investigators  conducted  interviews 
on  February  11,  1976.    Initial  documentation  was  received  as  follows: 

1.  Schedule  of  Services  and  Membership  Dues  (December,  1975) 

2.  By-laws,  Sound  Health  Association  (recorded  January  18,  1973) 

3.  Results  of  Survey  of  SHA  members  (August  -  September  1975) 

4.  SHA  Member  Questionnaire 

5.  Minutes  of  SHA  Fourth  Annual  Meeting  (January  29,  1975) 

6.  Quality  Assurance  Systems  (revised,  February  1975) 

7.  Minutes,  Health  Benefit  Committee  Meeting  (September  17,  1974) 

8.  Qualitative  and  Quantitative  Standards  for  Various  Diagnostic, 
Therapeutic  and  Laboratory  Services  (1975) 

9.  SHA  Health  Maintenance  Delivery  System  Flow  Diagram 

10.  SHA  Medical  Record 

11.  Statement  of  Income  and  Expenses  (September  30,  1975  -  quarterly) 

12.  Quality  Assurance,  Structure,  Process  and  Outcome  (August  22,  1973) 

13.  Quality  Assurance  Policy  (December  12,  1972) 

Documentation  outlines  the  goals,  objectives,  and  general  program  elements 
of  the  SHA  delivery  system  and  quality  assurance  program. 

A  significant  amount  of  information  was  gathered  on  the  operational  status 
and  direction  of  SHA's  quality  assessment  program  during  the  site  visit 
interviews.    Questions  were  administered  to  two  SHA  personnel ,  the  Executive 
Director  and  the  Medical  Director. 

Several  topic  areas  were  covered  during  the  questioning: 

1.  Status  of  SHA's  total  operation 

2.  SHA  complying  with  Federal  HMO  quality  assurance  regulations 
(Section  110.108  (J) 

3.  Responsibilities  of  medical  director,  assistant  director  in 
quality  assurance 

4.  Problem  of  implementing  quality  assessment  in  a  developing 
program 

5.  Specific  operational  quality  assurance  activities  at  SHA 

6.  Use  of  management  information  system  in  quality  assurance 

7.  Data  base  for  quality  assurance 

8.  Patient  satisfaction  questionnaires 

9.  Future  plans  for  quality  assessment  at  SHA 

10.  Philosophical  orientation  of  SHA  personnel  vis-a-vis  quality 
assurance 

11.  Relationship  between  medical  staff  and  administrative  staff 
vis-a-vis  quality  assurance 
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BLUE  SHIELD  OF  CALIFORNIA 
San  Francisco,  California 


SECTION  I 
INTRODUCTION 


Blue  Shield  of  California  (BSC)  in  San  Francisco  is  a  health  care  plan 
which  covers  individuals  for  physician  services  and  other  medical  expenses. 
BSC  also  operates  as  a  fiscal  intermediary  for  the  provision  of  Title  XVIII 
and  XIX  services  in  California.    (Title  XIX  services  are  administered  in  con- 
junction with  Blue  Cross).    Being  a  private  fiscal  intermediary,  BSC  has 
developed  and  implemented  various  quality  review  procedures  which  focus  on 
monitoring  utilization  of  ambulatory  physician  services. 

This  paper  will  briefly  describe  the  organizational  units  and  related 
functions  involved  in  performing  utilization  and  quality  review,  in  particular 
the  activities  of  the  Provider  Review  Division  (PRD)  which  reviews  submitted 
claims  for  payment.    This  Division  was  formed  in  1967  to  begin  utilization 
monitoring  of  Title  XVIII  and  XIX  and  CHAMPUS  claims.    The  PRD  originally  had 
a  staff  of  three  and  has  since  expanded  to  approximately  100  employees.  This 
study  examines  the  claims-based  postpayment  review  portion  of  review  activi- 
ties.   Other  activities  for  prepayment  review  will  be  described  only  peri- 
pherally in  context  with  their  relationship  to  postpayment  review. 

The  PRD  consists  of  four  organizational  components: 

1 •    Utilization  Audit  and  Review  Section 

2.  Medical  Policy  Section 

3.  Medical  Review  Section  (administrative  support  for  medical 
advisors) 

4.  Medical  Advisors 

The  Utilization  Section  is  responsible  for  administration  of  claims  review 

and  uses  data  reports  generated  in  the  Claims  Division,  although  the  processing 

includes  review  programs  developed  and  entered  by  the  PRD. 

They  are  responsible  for  implementing  the  computer  programs  for  review, 
preparing  individual  claims  and  cases  for  medical  review  and  correlating 
review  decisions  to  necessary  sources. 

The  Medical  Policy  Section  is  responsible  for  medical  decision-making 
for  those  cases  identified  by  the  Utilization  Section.    This  section  consists 
of  medical  advisors  and  is  administered  by  a  medical  director,  who  has  respon- 
sibility for  the  selection,  reviewing  and  rotation  of  the  advisors  based  on 
recommendations  from  local  medical  societies  throughout  California. 

The  Medical  Review  Administrative  Section  provides  staff  support  for 
the  medical  advisors,  who  review  the  exceptions  generated  by  the  computerized 
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audits  in  the  Utilization  Section.    Section  personnel  compile  the  results  of 
physician  review,  contact  outside  sources  regarding  the  review  and  schedule 
review  sessions  for  specific  advisors. 

The  Claims  Division  of  BSC  processes  all  medical  claims  entered  for 
payment.    Before  claims  go  to  the  Provider  Review  Division  they  are  processed 
through  the  claims  shops,  which  are  divided  into  four  sections.    If  profes- 
sional review  is  necessary,  claims  are  forwarded  to  the  Medical  Advisor 
Preparation  (MAP)  units. 


230-045  O  -  77  -  1 
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SECTION  II 


QUALITY       ASSURANCE  PROGRAM 


BACKGROUND 

During  the  two  site  visits  relating  to  this  study,  BSC  was  in  the  process 
of  changing  computerized  utilization  review  programs.    The  Peer  Group  Norm 
(PGN)  system  has  been  in  operation  the  longest  period  of  time,  although  it  is 
being  replaced  with  the  Retrospective  Analysis  of  Medical  Services  System 
(RAMS),  which  began  in  January,  1976.    As  of  April,  the  PGN  was  still  the 
only  program  fully  operational. 

The  Surveillance  Utilization  Review  (SUR)  system,  designed  by  the  Social 
Rehabilitation  Service,  functions  concurrently  with  PGN.    Supported  through 
federal  funding,  this  system  is  oriented  specifically  for  review  of  Title  XIX 
claims  and  is  not  yet  fully  operational;  therefore  this  section  describes  the 
PGN  system  in  more  detail.    Since  both  systems  are  essential  in  the  BSC 
postpayment  review  process,  the  description  will  attempt  to  picture  adequately 
the  interrelationship  of  the  two  systems  within  one  larger  system. 

BSC  personnel  said  the  PGN  system  was  initially  developed  by  the  EDS 
Federal  Corporation.    The  BSC  role  was  to  explicitly  define  the  end  results 
of  a  Utilization  Review  (UR)  system,  and  allow  EDS  to  develop  system  com- 
ponents.   BSC  provided  EDS  with  diagnostic  topics  and  appropriate  guidelines 
to  be  included  in  the  system.    Presently,  BSC  personnel  manage  all  UR  data 
processing. 

PGN  provides  BSC  with  practice  patterns  for  providers  who  exceed  a 
statistical  computed  norm  for  procedures  performed  and  paid  for  during  one 
fiscal  quarter.    Documents  define  peer  groups  as  the  physicians  of  a  com- 
parable specialty  within  a  PSRO  geographic  locale.    BSC  personnel  indicated 
approximately  150  individuals  occupy  each  peer  group.    This  program 
supplies  two  basic  reports:    (a)  the  080  Report,  which  provides  the  computed 
practice  patterns  for  providers  and  (b)  the  155  Report,  which  provides  BSC 
utilization  analysis  and  a  detailed  payment  history  for  providers  who  generate 
norm  variances. 

A  weighting  system  is  used  in  080  reports  to  measure  to  what  degree  a 
provider  exceeds  the  norm.    Both  080  and  155  reports  are  generated  quarterly, 
although  separate  data  elements  are  used  for  producing  the  reports.    The  080 
Report  elements  include:    physician  supplier  license  number  and  location, 
patient  count,  PSRO  area,  specialty,  exception  code  (those  who  exceeded  norm), 
type  of  procedure,  dollar  amount,  service,  norm,  individual  ratio,  flag  and 
weight.    Information  from  this  report,  once  reviewed  by  an  analyst,  will  be 
used  to  request  a  155  Report  for  a  provider.    The  080  Report  allows  the 
analyst  to  review  cases  based  on  a  weighted  sequence  since  the  heavier  weights 
will  be  reported  first,  in  rank  order. 
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Once  a  155  Report  is  requested  for  a  provider,  the  analyst  will  begin 
the  case  preparation  stage.    The  155  Report  elements  include:  provider 
license  number  and  name,  PSRO  area,  specialty,  patient's  name  and  I.D.  number, 
date  of  service,  diagnosis  code,  type  of  service,  claim  number,  explanation 
of  benefits  (coded),  check  number,  date  paid,  allowance  for  payment  and  billed 
and  total  line  numbers  (claims,  services,  patients). 

BSC  documents  indicate  the  above  computer  programs  and  audits  can  measure 
deviations  from  levels  of  care  based  on  criteria  established  by  peer  group'. 
BSC  uses  computers  "as  another  tool  or  arm  of  the  medical  advisors"  by  taking 
advantage  of  established  technology  in  establishing  and  documenting  level  of 
care  guidelines  and  automating  the  measurement  of  care.    This  measurement  is 
integrated  with  the  judgment  of  BSC  professionals  to  decide  (control  and 
monitor)  the  probity  of  a  particular  instance  of  care. 

The  PGN  080  and  155  Reports  are  the  essence  of  the  postpayment  review 
system.    Several  steps  occurred  in  the  development  of  these  specific  reports. 
The  first  step  required  processing  all  claims  data  received  by  BSC  (statewide 
and  with  PSRO  area)  to  produce  a  frequency  distribution  of  all  procedures. 
The  second  step  required  BSC  professionals  to  develop  acceptable  exception 
points  for  diagnoses,  which  were  then  programmed  as  computer  audits.  BSC 
personnel  indicated  once  a  frequency  distribution  is  produced  per  100  patients, 
BSC  then  processes  all  providers  against  these  levels.    In  terms  of  specific 
standard  development,  possible  criteria  were  generated  for  all  procedure  num- 
bers and  peer  groups  by  BSC  staff.    These  were  concurrently  reviewed  by  the 
BSC  medical  director  and  specialty  advisors.    BSC  personnel  indicated  there 
were  some  problems  in  reaching  agreement  on  exception  standards.    Once  the 
medical  director  approved  the  criteria,  they  were  entered  into  computer  pro- 
grams.   These  computerized  standards  are  subject  to  change  by  BSC  professional 
personnel  and  criteria  can  be  revised  through  an  informal  review  process  involv- 
ing a  senior  utilization  analyst  and  the  medical  advisors,  who  have  the  author- 
ity to  request  changes  in  criteria.    BSC  personnel  could  not  provide  informa- 
tion on  the  frequency  of  changes  in  established  criteria. 

The  final  step  is  the  actual  reporting  of  procedure  exceptions  by  pro- 
viders (in  rank  order).    Documents  note  this  report  is  generated  quarterly, 
using  a  three-month  claims  data  base.    Documents  suggest  two  types  of  weighting 
schemes  are  used,  one  reflecting  exceptions  based  on  noncompliance  to  total 
procedures  criteria,  the  other  reflecting  exceptions  within  individual  treat- 
ment norms.    The  080  Report  is  weighted  only  for  the  degree  to  which  an 
individual  provider  exceeds  the  total  norm,  not  an  independently  computed 
norm  for  his  practice  pattern. 

The  production  of  the  080  Report  is  the  first  step  in  the  four-step 
process  of  postpayment  utilization  review.    These  four  steps  include:  (a) 
case  identification,  (b)  case  preparation,  (c)  medical  advisor  review  and 
(d)  case  disposition. 

The  080  Report  identifies  cases  that  should  be  professionally  reviewed 
by  BSC  medical  advisors.    The  second  step  is  case  preparation  conducted  by  a 
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utilization  analyst.    Computerized  audits  are  not  the  only  sources  of  input 
in  determining  what  cases  should  be  reviewed.    Providers  to  be  reviewed  were 
identified  from  three  basic  sources:    (a)  computer  audits  (50%),  (b)  medical 
advisors  criteria  (25%),  and  (c)  federal  and  state  government  input  (25%). 

The  case  preparer  essentially  compiles  original  documentation  for  a 
specific  claim  after  noting  the  largest  weighted  norm  variances.    This  preparer 
reviews  exceptions  and  weights  from  the  080  Report,  and  compares  those  data 
with  the  profile  history  data  from  the  155  Report.    The  analyst  then  decides 
whether  or  not  to  forward  the  case  for  medical  advisor  review.    The  case 
preparer  also  decides  what  original  documentation  to  collect  in  putting  to- 
gether case  information.    Once  all  the  information  has  been  compiled,  the 
analyst  will  write  a  summary  to  forward  to  the  medical  advisor.    One  analyst 
indicated  she  sees  "providers  doing  the  same  things,  over  and  over,"  suggest- 
ing that  patterns  of  practice,  based  on  computerized  audits,  can  be  derived 
by  the  analyst.    This  fact  is  important  given  the  decision  that  the  analyst 
makes  in  the  total  review  process. 

The  case  preparer  keeps  a  log  of  all  cases  prepared  and  sent  for  review. 
After  90  days,  the  analyst  will  follow  up  the  case  to  find  its  disposition. 
The  analyst  indicated  approximately  750  cases  are  prepared  quarterly. 

The  third  step  involves  implicit  peer  review  by  BSC  medical  advisors. 
Physician  advisors  review  all  documentation  provided  to  determine  the  probity 
of  care,  and  determine  whether  the  weighted  exceptions  were  acceptable  based 
on  valid  medical  opinion. 

This  component  of  BSC  review  process  is  the  most  critical.  Documents 
outline  several  options  a  medical  advisor  can  implement: 

T.    The  practice  is  acceptable,  no  further  action  is  necessary 

2.  This  is  a  borderline  case,  review  another  quarter 

3.  Request  more  information  from  provider  either  in  writing 
or  by  phone 

4.  Call  provider  in  for  a  conference 

5.  Place  provider  on  prepayment  review  for  possible 
presentation  to  local  review  committee 

6.  Recover  paid  claims  for  a  period  of  time  and  then  go 
to  the  local  review  committee 

7.  Possible  fraud  is  discovered  and  the  matter  is  turned 
over  to  the  proper  authorities. 

Options  (1)  through  (4)  are  conducted  solely  by  the  advisor  or  in  con- 
junction with  the  case  preparer.    Option  (5)  involves  kicking  out  all  a 
provider's  claims  forwarded  from  the  Claims  Division  to  medical  advisors 
for  review.    Option  (6)  involves  a  more  detailed  approach.    During  interviews, 
BSC  personnel  indicated  one  senior  program  analyst  is  responsible  for  pre- 
paring and  scheduling  case  presentations  to  local  County  Medical  Society  Peer 
Review  Committees.    The  analyst  substantiated  the  case  for  BSC  by  documenting 
claims  information  (basis  of  exception)  with  the  assistance  of  the  case  pre- 
parer.   This  entails  the  collection  of  additional  source  material,  including 
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the  medical  record,  X-rays,  and  laboratory  records.    BSC  notifies  a  local  committee 
in  writing  that  it  wishes  to  present  a  case.    At  the  same  time,  BSC  notifies  the 
physician  in  question.    The  medical  advisor  who  made  the  decision  to  forward  the 
case  to  committee  (after  review  by  BSC  medical  director)  is  not  required  to  attend 
the  meeting,  although  the  BSC  field  representative  does  attend.    The  local  commit- 
tee assigns  a  case  manager  who  reviews  the  case  and  presents  his  opinion  to  the 
committee.    The  provider  in  question  has  the  opportunity  to  present  his  version 
of  the  case,  and  the  committee  comes  to  a  decision  in  a  closed-door  meeting. 
Local  committee  decisions  usually  consider  BSC  recommendations  in  conjunction 
with  the  other  evidence,  and  have  five  choices  of  corrective  action:    (a)  post- 
pone decision,  (b)  concur  with  BSC,  (c)  adjust  physician  RVS  billing  pattern, 
(d)  deny  payment  for  services,  (e)  indicate  overutil ization. 

For  option  7  (see  previous  page)  BSC  refers  cases  to  authorities  in  proper 
governmental  agencies  for  investigation  and  possible  prosecution. 

Because  of  the  importance  of  the  medical  advisor's  reviews  in  the  BSC  utili- 
zation review  program,  BSC  personnel  were  questioned  on  procedures  for  choosing 
medical  advisors.    Each  medical  advisor  is  nominated  by  a  local  medical  society 
on  request  from  BSC.    Advisors  must  be  in  active  private  practice,  and  must  be 
willing  to  perform  review.    BSC,  after  reviewing  nominations,  contacts  providers 
to  ascertain  interest  in  conducting  reviews.    If  a  physician  decides  to  become 
a  medical  advisor,  BSC  provides  orientation  to  review  procedures.    This  entails 
an  introduction  to  BSC  review  philosophy  and  method,  and  a  "practice"  review  of 
claims  with  the  assistance  of  an  experienced  review  person  (usually  a  registered 
nurse).    The  review  person  observes  the  nominated  physician  and  forwards  comments 
to  the  BSC  medical  director.    Advisors  do  not  have  a  contractual  agreement  with 
BSC,  but  are  employed  as  consultants.    It  was  noted  that  advisor  reviews  are 
evaluated  by  a  variety  of  BSC  personnel,  with  the  final  decision  resting  with 
the  BSC  medical  director.    BSC  employs  approximately  200  advisors;  most  work  less 
than  20  hours  per  week. 

Step  four  in  the  postpayment  system  requires  disposition  of  review  cases. 
The  medical  advisors  will  accept  or  not  accept  the  reported  exception.  Non- 
accepted  cases  go  to  the  Medical  Review  Section  for  discussion  of  the  case 
problem,  and  a  decision  as  to  whether  to  place  a  provider  on  prepayment  review 
is  made. 

The  PGN  review  mode  has  been  used  for  four  years;  evidence  collected  suggests 
it  has  been  put  in  operation  as  planned.    BSC  personnel  indicated  the  Surveillance 
Utilization  Review  System  (SUR)  will  be  implemented  and  operate  concurrently  with 
the  PGN  system. 


Surveillance  Utilization  Review  System  (SUR) 

The  SUR  was  scheduled  for  implementation  beginning  January  1,  1976,  but 
was  not  yet  fully  functional  at  the  time  of  the  site  visits  and  therefore  will 
not  be  fully  described.    The  SUR  is  similar  to  the  PGN  program;  federal  funding 
is  available  to  those  review  organizations  (Title  19  fiscal  agents  for  states) 
who  implement  the  SUR. 
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The  SUR  is  retrospective  and  oriented  toward  u til izati on  patterns  on  a 
participant  (provider- beneficiary)  basis.    As  in  the  PGN  system,  norms  are  com- 
puted to  represent  the  range  of  reasonable  participant  utilization  for  specific 
diagnostic  and  treatment  categories  within  specialty  groups.    Individual  activity 
is  compared  to  the  norms  and  exceptions  are  then  identified  for  further  review. 

A  major  aspect  of  the  SUR  is  the  exception  processing  apparatus.    The  sum- 
mary profile  reports  are  implemented  through  automated  exception  processing 
techniques.    The  exception  processing  requires  six  major  steps: 

1.  A  statistical  profile  is  developed  from  information  contained  in  paid 
claims  for  each  individual  provider  and  recipient 

2.  The  statistical  profiles  of  all  providers  and  recipients  are  classified 
into  peer  groups 

3.  Averages  and  standard  deviations  are  computed  by  peer  group  for  each 
indicator  contained  within  each  statistical  profile 

4.  The  averages  and  standard  deviations  computed  in  step  three  are  used 
to  establish  norms  or  exception  criteria  by  peer  group  for  each  indi- 
cator in  a  statistical  profile 

5.  The  statistical  profile  of  all  individual  providers  or  recipients 
within  each  class  group  is  evaluated  indicator  by  indicator  against 
the  matching  exception  criteria  established  for  each  peer  group 

6.  All  providers  or  recipients  from  at  least  one  exception  noted  during 
step  five  are  printed  out  for  manual  review 

Three  levels  of  reports  are  generated: 

1.  Management  Summary  Reports,  which  reflect  the  performance  of  a 

recipient  and  provider  group  based  on  a  comparison  of  utilization 
with  specialty  class  norms 

2.  Summary  Profile  Reports,  which  detail  recipients  and  providers  who  have 
exceeded  norms 

3.  Claims  Detail  Reports,  which  reflect  detailed  claim  information  and 
are  available  on  request 

In  terms  of  functional  reporting,  the  system  is  divided  into  two  major 
categories:  recipient  reporting  and  provider  reporting.    Both  categories  have 
major  subsystem  functions  with  the  results  being  generated  in  Management  Summary 
Reports  and  Profile  Reports. 

The  provider  subsystem  begins  with  the  processing  and  storing  of  history 
data  in  three  separate  files,  then  continues  by  computing  norms  and  selecting 
exceptional  claims  which  are  also  placed  in  files;  the  other  files  in  the  sys- 
tem consist  of  provider  summaries  and  peer  group  summaries.    This  summary  of 
provider  performance  is  reviewed  with  the  class  group  and  processed  through  the 
exception  file.    The  Provider  Summary  Profile  is  produced  after  formatting.  The 
exception  claim  file,  after  formatting,  produces  the  exception  claim  detail 
reports.    The  same  basic  program  is  followed  for  the  recipient  reports,  with  the 
important  addition  of  the  eligibility  master  file  being  processed. 

Although  the  present  BSC  utilization  review  system  (PGN)  will  operate  con- 
currently with  SUR,  there  seem  to  be  several  differences  between  the  systems: 
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BSC  Utilization  Review  (PGN) 


SUR 


1.  Specific  procedures  are 
processed 

2.  Reports  produced  quarterly 

3.  No  trend  data 

4.  Six-month  data  base 

5.  Procedures  flexible  for 
programming  changes 

6.  Selects  out  any  provider 
desired  by  analyst 

7.  Uses  payment  date  to 
begin  processing 


1.  Procedure  categories  are 
processed 

2.  Reports  produced  monthly 

3.  Trend  data 

4.  Fifteen-month  data  base 

5.  Procedures  categories  not 
flexible  for  change 

6.  No  method  to  pick  out  any 
providers 

7.  Uses  service  data  to  begin 
processing 


Retrospective  Analysis  of  Medical  Services  (RAMS) 

The  final  area  involved  with  postpayment  utilization  review  deals  with 
BSC's  implementation  of  Retrospective  Analysis  of  Medical  Services  (RAMS),  a 
model  treatment  program  which  evaluates  services  (based  on  claims  data)  for  a 
specific  problem  or  diagnosis  in  comparison  with  an  ideal  or  expected  patterns 
of  care.    BSC  personnel  indicated  RAMS  was  implemented  January  1,  1976,  but 
it  was  too  early  to  collect  any  data  on  its  operational  status.    This  system, 
developed  by  EDS  Federal  Corporation,  produces  (a)  periodic  histories,  profiles 
and  exception  reports  based  on  diagnoses,  (b)  subsystem  programming  for  special 
studies  as  requested  and  (c)  detailed  provider  and  patient  histories. 

Data  taken  from  claims  information  may  be  divided  into  four  categories: 

1.  Geographic  locality  according  to  PSRO  area,  statewide  or  regional 
parameters 

2.  Specialty 

3.  Diagnosis 

4.  Type  of  service  or  procedure 

The  main  measurement  tool  of  this  system  is  the  development  of  treatment 
models  with  defined  paradigms  for  "required,"  "optional,"  or  "never,"  which  note 
when  certain  services  are  to  be  provided  for  a  specific  diagnosis.    The  treatment 
models  can  be  developed  in  five  categories: 

1.  Analysis  of  existing  diagnosis  and  treatment  interrelations 

2.  Review  of  the  above  reports  is  incomplete;  domain  of  the  analysis  is 
selected  by  specifying  diagnostic  procedural  categories 

3.  Empirical  treatment  models,  based  on  past  events,  historical  data 

4.  Models  can  be  modified  by  user  to  form  ideal  models 

5.  Norms  are  computed  for  each  procedure  cateogry  within  each  diagnostic 
category,  and  are  consistent  with  the  treatment  models 

This  system  generates  lists  of  exceptions  to  the  established  model  treat- 
ments.   The  exceptions  are  reported  for  both  providers  and  recipients,  and  the 
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system  can  combine  groups  and  rank  them  in  order  based  on  frequency  of  excep- 
tions.   This  reporting  capability  deals  with  the  summary  profile  reports.  Once 
a  category  of  service  is  chosen  (which  may  be  different  from  this  in  treatment 
analysis),  it  can  be  defined  by  combining  procedure  codes  and  providers  into 
eight  major  categories,  using  any  combination  of  procedures.    The  capability 
exists  for  eight  major  categories,  ultimately  obtaining  ratios,  percentages, 
etc.,  and  reflecting  exceptions. 

This  sytem  uses  a  15-month  data  base,  and  data  can  be  retrieved  and  reported 
for  any  specified  period  within  that  15-month  time. 


ADDITIONAL  COMMENTS 

Blue  Shield  of  California,  as  the  Title  XIX  fiscal  agent  for  the  State  of 
California,  is  responsible  for  collecting,  processing,  and  reviewing  Medicaid 
claims  for  the  entire  state.    This  responsibility  requires  BSC  to  implement 
automated  data  systems  which  support  the  required  professional  review  functions. 
The  system  is  presently  undergoing  change  because  of  the  implementation  of  SUR 
and  RAMS;  however,  the  professional  review  conducted  by  the  medical  advisors 
remains  unchanged,  and  is  the  cornerstone  of  the  total  review  process. 
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SECTION  III 
HCMS     STUDY  INFORMATION 


Blue  Shield  of  California  was  included  in  study  at  the  suggestion  of  the 
Study  Advisory  Board  in  January,  1976,  to  include  a  fiscal  intermediary  which 
reviewed  claims  for  utilization  and  quality  control.    Staff  contacted  BSC  to 
ascertain  interest,  and  they  agreed  to  participate  in  a  telephone  discussion 
on  February  10,  1976.    Initial  documentation  of  BSC  review  activities  was  re- 
ceived on  February  23,  1976.    Documents  provided  included  the  following: 

1.  Blue  Shield  Post-Payment  Utilization  Review  (summary  document) 

2.  Surveillance  and  Utilization  Review  (SUR)  manual 

3.  Peer  Group  Norm  Report  (Revised  October  1,  1974) 

4.  Utilization  Review  System,  Users  Manual  (EDS) 

5.  Retrospective  Analysis  of  Medical  Services:    A  Treatment  Model 
Approach  (December,  1975) 

Further  information  regarding  the  BSC  Specific  Utilization  Review  system 
was  collected  during  two  site  visits,  March  5  and  April  2,  1976.    Several  per- 
sonnel were  interviewed  including: 

1.  Senior  Utilization  Analyst 

2.  Utilization  Analyst  and  Case  Preparer 

3.  Senior  Program  Analyst 

4.  Assistant  Vice  President,  Provider  Review  Division 
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UTAH  PROFESSIONAL  REVIEW  ORGANIZATION 
PHYSICIAN  AMBULATORY  CARE  EVALUATION  SYSTEM 


Salt  Lake  City,  Utah 
SECTION  I 
INTRODUCTION 


The  Utah  Professional  Review  Organization  (UPRO)  is  responsible  for 
performing  quality  review  of  ambulatory  care*  provided  through  the  Utah 
Medicaid  Program  under  contract  with  the  Utah  Department  of  Social  Services. 
The  Physician  Ambulatory  Care  Evaluation  Module  (PACE)  was  designed  by 
UPRO  and  Optimum  Systems,  Inc.  (OSI)  to  detect  problems  in  the  type  of 
ambulatory  care  provided  by  physicians  through  the  use  of  automated  screen- 
ing methods.    Data  used  for  review  are  generated  by  the  Medicaid  claim  forms 
collected  and  preprocessed  by  the  Utah  Medicaid  Program.  The  review  system 
is  predicated  on  the  use  of  automated  data  processing  to  capture  necessary 
medical  data  from  claim  forms  submitted  to  the  State  of  Utah  for  payment. 
The  Medicaid  Management  Information  System  and  PACE  claims-based  review 
system  compares  this  information  with  criteria  or  screening  parameters  estab 
lished  by  the  UPRO  Quality  Evaluation  Committee.    Profiles  which  contain 
instances  of  questionable  care  are  reviewed  by  physician  reviewers.  The 
physician  reviewers  take  action  based  on  comparison  of  criteria  and  services 
Where  patterns  of  care  appear  unjustified,  the  PACE  Ambulatory  Care  Review 
Committee  contacts  physicians. 


*  UPRO  is  also  responsible  for  review  of  hospital  care 
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SECTION  II 


QUALITY     ASSURANCE  PROGRAM 


HISTORY  AND  FUNDING 

In  1972,  Public  Law  92-603  created  the  Professional  Standards  Review 
Organizations  (PSROs),  which  were  designed  to  assess  medical  care  and  reduce 
the  costs  of  care  provided  under  Titles  V,  XVIII  and  XIX.    At  the  same  time, 
federal  funds  were  made  available  for  the  development  of  management  infor- 
mation systems  to  improve  state  level  administration  of  Title  XIX  throughout 
the  country. 

The  Utah  Medicaid  program  initiated  in  July,  1966,  was  one  of  the  first 
state  programs  established  under  Title  XIX.    According  to  documentation,  the 
program  provides  services  to  approximately  77,000  individuals  each  year  at  an 
approximate  cost  of  $25,000,000.    Because  of  the  large  demand  for  services  by 
the  population,  the  existing  system  had  difficulty  administering  and  controlling 
program  costs. 

To  improve  the  capability  of  the  Title  XIX  State  agency  to  administer  the 
Medicaid  program,  the  Utah  Department  of  Social  Services  established  a  project 
which  provided  for  the  development  and  implementation  of  a  Utah  Medicaid 
Management  Information  System  (UTAH  MMIS).    In  seeking  new  processing  capa- 
bilities, it  hoped  to  incorporate  in  the  program  efficient  professional  review 
by  UPR0. 

The  Utah  Professional  Review  Organization  (UPR0)  was  supported  initially 
by  the  Utah  State  Medical  Association,  and  subsequently  by  an  EMCRO  grant  from 
the  DHEW  Bureau  of  Health  Services  Research  and  Development.    The  PACE  program 
was  initially  developed  to  demonstrate  the  feasibility  of  performing  evaluations 
of  medical  care  by  examining  data  available  only  on  claim  forms.    Initial  con- 
cepts were  developed  by  UPR0  personnel;  subsequently  several  firms  (University 
of  Utah,  Dikewood,  0SI,  Blue  Cross)  had  bid  for  the  contract  to  implement  initial 
PACE  design.    The  contract  was  awarded  to  0SI,  who  had  formulated  a  data  proces- 
sing component  for  PACE.    On  October  1,  1972,  an  agreement  was  reached  between 
UPRO  and  the  Utah  Department  of  Social  Services  to  make  Medicaid  claims  avail- 
able to  OS  I  for  entry  into  PACE  under  the  EMCRO  grant.    At  this  time,  Utah  did 
not  have  an  automated  claims  data  entry  process,  so  OSI  performed  the  functions 
of  medical  encoding,  claims  data  entry  and  editing.    Soon  after  this  agreement, 
data  input  and  the  screening  of  claims  data  against  developed  medical  criteria 
began.    A  major  part  of  the  EMCRO  project  was  the  initial  development  of 
medical  criteria  to  screen  the  Title  XIX  claims. 

By  early  1974,  the  original  concept  had  been  tested  and  proved  feasible. 
At  this  time,  the  idea  of  a  package  which  could  perform  the  function  of  PACE 
and  could  be  integrated  with  planned  or  existing  MMIS  operations  was  formulated. 
The  knowledge,  experience  and  cooperation  developed  among  UPRO,  the  Utah  State 
Department  of  Social  Services  and  OSI  led  to  a  contract  with  the  Social  and 
Rehabilitation  Services  (SRS)  in  July,  1974  for  the  development  of  the  MMIS-PACE 
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Module.    The  PACE  system  had  been  developed  and  operational ized  by  UPRO 
during  the  EMCRO  contract  period  in  order  to  review  Title  XIX  care.  When 
integration  was  proposed,  the  PACE  module  was  already  in  operation.  It 
was  necessary  to  design  software  to  link  PACE  with  MMIS  and  OSI  was  charged 
with  the  responsibility  of  developing  a  refined  software  module  compatible 
with  the  Surveillance  and  Utilization  Review  (SUR)  component  of  Utah  MMIS. 
UPRO's  main  activity  during  this  time  was  to  identify  reporting  and  screening 
logic  needs  for  the  integrated  system.    By  October,  1975,  the  new  software 
was  installed  and  PACE  was  successfully  integrated  with  MMIS. 

UPRO  personnel  indicated  the  core  of  their  review  methodology  lies  in 
development  and  use  of  automated  medical  guidelines.    These  guidelines  produce 
exception  reports  which  are  reviewed  by  UPRO  physicians.    Individual  guide- 
lines (computerized  files)  are  periodically  revised  by  the  Ambulatory  Care 
Review  Committee.    Since  the  guidelines  are  computerized,  UPRO  staff  (in 
conjunction  with  OSI)  can  quickly  assess  and  revise  them.    UPRO  personnel 
indicated  guideline  development  and  programming  had  been  a  complicated 
process  requiring  substantial  investment.    Guidelines  are  continually  updated. 

Since  PACE  is  essentially  designed  to  compare  claims  data  with  established 
criteria  and  report  exceptions,  the  professional  assessment  is  governed  by 
UPRO's  approach  to  review,  which  is  twofold: 

a.  to  review  exceptions  in  light  of  what  educational  needs  a 
particular  physician  requires 

b.  to  review  and  make  professional  judgment  based  on  a  pattern 
or  trend  of  exceptions,  rather  than  isolating  one  case  and 
basing  action  on  that 

UPRO  views  its  role  as  one  of  liaison  with  the  entire  medical  community. 
As  a  representative  of  that  community  in  a  professional  review  effort,  UPRO 
fills  this  role  by  attempting  to  identify  and  involve  physicians  supportive 
of  review  efforts.    UPRO  emphasizes  solid  relationships  with  the  physician 
community  because  review  system  success  depends  on  physician  participation. 
Although  the  entire  review  system  is  based  on  automated  data  processing, 
the  importance  of  UPRO's  philosophical  approach  to  professional  review 
ultimately  decides  the  comprehensiveness  of  the  system. 


DOCUMENTED  PROGRAM  COMPONENTS 

According  to  June,  1975,  documentation,  the  overall  objective  to  the  Utah 
MMIS-PACE  project  was  to  develop  a  computerized  system  to  support  a  PSRO  in 
performing  ambulatory  care  review  for  the  Medicaid  program.    Within  this 
general  objective  the  MMIS-PACE  Module  was  designed  to  meet  the  following 
specific  objectives: 

1.    The  system  must  provide  effective  tools  for  reviewing  Medicaid 
ambulatory  care  claims  data  by  providing 
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a.  a  flexible  means  of  communicating  professional  criteria 
to  a  computer  system  (includes  modification  and  deletion 
as  needed) 

b.  the  ability  to  perform  prepayment  review  immediately 

c.  the  ability  to  construct  patient  treatment  review  so 
quality  of  care  and  utilization  patterns  can  be  observed 
over  time 

2.  Develop  the  review  system  by  taking  advantage  of  the  review 
experiences  of  UPRO  and  others  who  have  been  involved  in  ambulatory 
care  review 

3.  Support  the  data  requirements  of  a  PSRO  by  providing 

a.  patient  treatment  profiles 

b.  other  requests  for  selective  information  from  the  data  base 

4.  Take  advantage  of  specific  knowledge  gained  by  the  UPRO  pilot 
project 

5.  Develop  a  relatively  simple  interface  to  the  Utah  MMIS  system 

6.  Provide  a  software  package  that  is  transferable  as  a  review  tool 
for  other  PSROs 

According  to  documentation,  analysis  of  the  quality  of  care  is  performed 
based  on  objective  criteria  (on  guideline)  developed  and  ratified  by  the  Quality 
Evaluation  Committee.    Each  specialty  is  responsible  for  criteria  development  in 
its  own  specialty  area.    Physician  reviewers  may  also  recommend  criteria  modifi- 
cation or  the  creation  of  new  criteria.    To  develop  criteria,  a  committee  member 
makes  a  recommendation  which  is  approved  or  denied  by  the  entire  committee.  The 
number  of  criteria  developed  depends  on  the  activity  of  the  committee.    The  cur- 
rent (July,  1976)  guidelines  set  includes  231  different  screening  guidelines  for 
114  different  categories  of  diagnoses,  procedures,  and  medications.    Drug  criteria 
have  been  developed  for  drug  data  processed  from  pharmacy  claims.    These  criteria 
are  guidelines  against  which  ambulatory  services  as  recorded  on  claim  forms  can 
be  compared  in  an  automated  screening  process.    Some  criteria  pertain  to  single 
encounters  and  other  analyze  a  patient's  history  of  care. 

Criteria  are  designed  to  identify  combinations  of  care  which  are: 

1.  Required  for  ideal  minimum  care 

2.  Inconsistent  with  ideal  care,  or 

3.  In  excess  of  reasonable  utilization  patterns 

Each  automated  guideline  has  a  number  of  keys  against  which  claims  data 
are  passed.    Certain  major  keys  determine  whether  new  claims  data  will  be  re- 
viewed against  specific  guidelines,  once  UPRO  receives  the  claims  data  from  the 
state.    There  are  two  levels  of  guideline  screening:    (a)  the  major  key  which 
triggers  guideline  review,  and  (b)  subordinate  keys  integrated  into  each  guide- 
line item.    The  first  level  keys  pass  claims  data  against  many  items,  including: 
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1.  Diagnosis  or  diagnosis  list 

2.  Provider  identification  and  specialty 

3.  Procedures  and  frequency  of  procedures 

4.  Drug  information  (code,  classification,  dosage) 

The  set  of  subordinate  audit  keys  include  the  following  items: 

1.  Object  of  search  (e.g.,  diagnosis,  procedure,  drug  reference) 

2.  Threshold  condition  (e.g.,  length  of  time  claim  should  be 
held  for  review  to  be  conducted) 

3.  Limit  section,  which  defines  the  (maximum  or  minimum)  number  of 
occurrences  for  objects  being  searched  or  reviewed 

Because  guidelines  are  the  critical  component  of  the  review  system,  they 
reflect  programming  interactions  that  will  capture  the  infinite  number  of  com- 
binations of  medical  data  elements  on  submitted  claim  forms.    The  keys  essen- 
tially search  for  the  proper  guideline  areas  necessary  to  screen  a  claim 
against  specific  combinations  of  diagnostic,  procedure,  provider  and  patient 
information. 

All  services  which  conform  to  established  criteria  (screening  parameters) 
are  not  subject  to  peer  review  and  are  cleared  for  immediate  payment.  Those 
services  not  conforming  to  established  criteria  are  reported  out  for  evaluation 
by  a  physician  reviewer.    The  PACE  Module  can  perform  two  separate  but  supportive 
levels  of  review. 

Prepayment  review  immediately  screens  all  claims  against  the  physician- 
developed  medical  guidel ines.  It  is  also  possible  for  a  reviewer  to  flag  all 
claims  entering  the  system  for  specific  patients,  providers,  diagnosis  codes, 
medication  codes,  etc.  Claims  which  pass  through  the  prepayment  review  cycle 
and  conform  to  computerized  criteria  are  automatically  given  a  "pay"  disposi- 
tion. Those  claims  failing  this  level  of  audit  are  suspended  for  physician 
review.  A  prepayment  evaluation  is  based  on  the  patient's  history  up  to  the 
time  the  claim  is  received. 

In  addition  to  being  evaluated  on  a  prepayment  basis,  a  claim  may  be 
placed  in  a  postpayment  evaluation  cycle  for  additional  evaluation  against 
time-dependent  criteria.    Postpayment  review  is  retrospective  and  examines 
care  and  treatment  patterns  over  time.    This  level  of  physician  review  con- 
sists of  an  evaluation  of  the  medical  necessity  for  services  viewed  over  time. 
This  level  of  review  requires  service  records  be  placed  on  a  recycling  file 
which  releases  the  record  for  evaluation  only  after  the  specified  time  for 
holding  the  record  is  passed.    The  postpayment  review  allows  for  closer  UPRO 
monitoring  of  patients  and  providers  for  purposes  of  medical  education.  The 
system  allows  prepayment  screens  to  be  established  or  adjusted  to  reflect  the 
findings  of  retrospective  review  regarding  exceptional  patterns  of  care. 

Beginning  in  August,  1976,  UPRO  physician  review  procedures  were  modified 
to  allow  concentration  on  patterns  of  exceptional  care.    Under  the  new  proce- 
dures, most  exceptions  identified  by  computer  screening  are  given  an  automatic 
"pattern  data  not  yet  sufficient  for  review"  disposition.    Once  a  pattern  of 
questionable  care  has  been  identified  (tentatively  set  of  3  exceptions  on  a 
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single  guideline  for  an  individual  provider),  the  patients'  profiles  are 
generated  for  physician  review.    Some  guidelines,  however,  continue  to  be 
reviewed  on  a  case  by  case  basis.    These  include  all  "report  only"  and  "pend" 
guidelines,  as  well  as  other  selected  guidelines  for  which  individual  case 
review  is  deemed  necessary. 


CLAIMS  PROCESSING  AND  PHYSICIAN  REVIEW 

The  data  base  information  processed  by  the  PACE  module  is  obtained 
from  four  types  of  claim  forms: 

1.  Physician  invoices 

2.  Outpatient  hospital,  laboratory  and  X-ray  invoices 

3.  Pharmacy  invoices 

4.  Clinic  invoices 

According  to  June,  1975,  documentation,  the  Utah  Medicaid  program  cur- 
rently processes  100,000  claims  per  month.    All  the  ambulatory  care  claims 
are  processed  by  the  MMIS-PACE  module. 

The  system  is  dependent  on  the  following  information: 

From  physician  claim  forms: 

a.  Date  of  service  including  period  of  hospitalization 

b.  Procedure  codes  and  descriptions 

c.  A  diagnosis  linked  to  each  procedure  (up  to  four  different 
diagnoses) 

d.  An  indication,  in  the  case  of  injections,  of  the  type  of 
medication  administered 

From  pharmacy  claim  forms: 

a.  Prescribing  and  dispensing  dates 

b.  The  identification  number  of  the  prescribing  physician 

c.  Drug  codes  in  the  case  of  commonly  prescribed  drugs 

d.  Quantity  prescribed 

From  outpatient  hospital  and  lab  invoices: 

a.  Dates  of  services 

b.  Outpatient  procedures  codes 

c.  Primary  and  secondary  diagnosis,  if  available  (not  linked 
to  specific  procedures) 


MMIS  PROCESSING  PRIOR  TO  PACE  INPUT 

Medicaid  claims  are  received  at  the  Utah  Department  of  Social  Services. 
The  original  and  one  copy  of  the  three-part  form  are  received  by  the  department 
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while  the  provider  retains  the  other  copy.    An  initial  check  is  performed  for 
completeness  by  the  claims  reviewers  in  the  Medical  Claims  Section.    If  neces- 
sary, incomplete  claims  are  returned  to  the  providers.    Claims  are  then  orga- 
nized into  batches  and  each  claim  is  assigned  a  document  control  number  and  a 
batch  identification.    After  document  numbers  are  assigned,  the  claims  carbon 
copies  are  separated  from  the  originals  and  sent  to  UPRO  as  hard  copy  evidence 
for  physician  review  and  periodic  accuracy  checks.    Data  entry  is  performed  by 
the  Department  of  Social  Services.    A  computer  audit  determines  the  eligibility 
of  the  claim  with  regard  to  the  provider,  patient  diagnosis,  etc.    Those  claims 
termed  ineligible  are  automatically  denied  payment  (reasons  for  denial  will 
accompany  the  remittance  statement)  and  are  not  included  in  the  PACE  data  base. 
The  information  to  be  sent  to  UPRO  for  PACE  review  is  copied  onto  the  PACE  Inter- 
face (or  Review)  Tape.    The  pharmacy,  outpatient,  clinic,  and  lab  claims  then 
proceed  to  payment.    Physician  claims,  however,  are  held  in  suspense  until  a 
clearance  for  payment  is  returned  on  a  PACE  Review  Disposition  Tape. 


PACE  COMPUTER  PROCESS 

After  UPRO  receives  the  tape,  it  is  sent  to  OSI  to  be  processed  into  the 
PACE  module.    Upon  receipt  of  the  tape,  a  second  provider  and  patient  eligi- 
bility check  is  performed.    The  claims  data  are  then  measured  against  compu- 
terized screening  parameters.    In  addition  to  the  diagnosis,  procedure  and  drug 
types  of  criteria  noted  previously,  the  module  also  includes  two  other  types  of 
criteria.    The  first  are  screening  guidelines  keyed  to  certain  provider  charac- 
teristics; the  second  type,  "report  only  criteria,"  are  tentative  guidelines 
which  are  in  the  process  of  being  investigated  for  inclusion  in  the  permanent 
guidelines.    The  number  of  exceptions  identified  for  each  "report  only"  guide- 
line are  tabulated  to  determine  if  a  sufficient  level  of  activity  exists  to 
justify  inclusion  of  the  guideline. 

As  a  result  of  the  computerized  screening,  at  least  five  outcomes  occur: 

1.  Claims  conforming  to  computerized  criteria  are  approved. 

2.  All  claims  data  are  added  and  updated  in  the  history  file  to 
maintain  the  integrity  of  the  data  base.    This  data  base  will 
serve  to  permit  the  continuation  of  the  utilization  and  quality 
analysis  process  and  will  also  be  the  source  of  supporting  data 
for  all  claims  identified  as  exceptions  in  the  criteria  screening 
process. 

3.  The  primary  tool  for  review  is  the  Patient  Profile  (which  prior 
to  August,  1976,  was  generated  automatically).    Each  profile 
contains  the  entire  patient  history  on  file,  with  descriptive 
messages  relating  each  flagged  treatment  to  the  cause  of  the 
exception,  the  date  flagged  and  the  review  date  and  disposition. 
Patient  profiles  are  constructed  to  provide  the  reviewer  with 
enough  information  to  make  a  case  judgment. 
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4.  All  exceptions  are  maintained  for  each  provider  automatically  in 
the  Exception  Message  File.    Each  time  an  exception  is  found  for 
a  specific  provider,  the  computer  automatically  adds  the  informa- 
tion to  the  already  established  exception  record  for  that  physi- 
cian in  addition  to  printing  it  out  on  an  exceptional  Patient 
Treatment  Profile. 

5.  Potential  violations  of  time-span  guidelines  are  added  to  the 
"Hold  File"  for  later  review.    They  are  rescreened  each  time 
claims  are  received. 

All  profiles  are  delivered  to  UPRO  for  professional  review.    In  addition 
to  these  profiles,  other  reports  (as  well  as  additional  information)  can  be 
requested  as  needed  by  the  physician  reviewers  or  the  members  of  the  Ambulatory 
Care  Review  Committee. 

Special  administrative  statistical  and  summary  reports  may  be  generated 
by  the  Review  Request  Submodule.    (This  is  one  of  five  submodules  which  consti- 
tute the  structure  of  the  MMIS-PACE  Module.    The  other  submodules  will  be  dis-  * 
cussed  in  greater  detail  later  in  the  description.)    It  is  capable  of  presenting 
information  in  several  ways  -  specific  patient  profiles  and  samples  as  well  as 
statistical  and  summary  reports. 

There  are  nine  summary  reports: 

1.  Suspense  Audit  Report  (PR-610):    Lists  all  claims  currently  in 
the  PACE  Suspense  File  (exceptions  for  which  no  disposition  has 
been  assigned).    The  report  is  sorted  by  patient  age  group  and 
ID  number,  and  flagged  by  date.    In  addition,  a  Suspense  Cross 
Reference  Report  (PR-611)  can  be  generated  to  provide  a  list  of 
all  providers  with  multiple  exceptions  currently  awaiting 
physician  review. 

2.  Line  Item  Summary  Report  (PR-624):    Provides  a  summary  of  the 
usage  of  services  in  association  with  diagnoses  and  a  summary 
of  the  occurrences  and  the  quantities  dispensed  of  drugs.  The 
lists  are  sorted  by  procedure  code  and  drug  code,  respectively. 

3.  Exception  Message  File  Report  (PR-630):    Provides  a  listing  of 
each  exception  message  on  file,  for  specified  periods.  Each 
report  can  be  coded  to  include  or  exclude  any  evaluation  dis- 
position code  and  up  to  10  provider  specialty  codes.    The  report 
can  be  sorted  by  any  or  all  of  the  following  fields  (in  any  order): 
specialty  code,  provider  I.D.,  date  of  service,  date  of  review, 
guideline-audit  number,  and  client. 

4.  Reviewer  Activity  Report  (PR-640):    Presents  summaries  of  each 
physician  reviewer's  activity,  sorted  by  guideline,  for  a  spe- 
cified period.    The  total  number  of  exceptions  evaluated  and  the 
disposition  distribution  is  presented  for  each  reviewer  for  each 
guidel ine. 
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5.  Provider  Performance  Profile  (PR-650):    Provides  a  summary  of 
Medicaid  services  provided  by  physicians,  and  a  tally  of  the 
number  of  times  each  of  those  services  was  flagged  as  an  excep- 
tion.   The  report  also  contains  normative  comparison  data  of 
services  and  exceptions  recorded  for  the  provider's  specialty 
and  by  all  physicians  in  the  provider's  county. 

6.  Provider  Exception  Summary  Report  (PR-680) :    Furnishes  a  summary 
of  exception  activity  by  provider  for  a  specified  time  period. 
The  report  is  sorted  by  provider  specialty  code,  by  provider 
and  then  by  guideline-audit  number. 

7.  Guideline  Exception  Summary  Report  (PR-681):    Furnishes  a 
summary  of  exception  activity  by  guideline  and  audit  number 
for  a  specified  time  period.    The  information  contained  in 
PR-681  is  the  same  as  in  PR-680;  however,  it  is  sorted  first 
by  guideline-audit  number  and  then  by  specialty  code  and 
provider. 

8.  Guideline  Report  (PR-690):    Provides  a  description  of  the  guide- 
lines active  in  the  PACE  Criteria  Evaluation  System  at  any  point. 
The  report  is  a  narrative  description  of  each  audit  which  has 
been  defined  for  each  guideline. 

9.  Guideline  Description  Report  (PR-691):    Contrary  to  its  title, 
this  report  provides  a  list  of  all  guidelines  sets  and  their 
associated  audits.    It  does  not  describe  the  guidelines  or  audits 
in  narrative  form. 

In  addition,  special  studies  may  be  requested  by  a  physician  reviewer, 
a  review  committee  or  a  specific  committee  administrator.    They  may  be  moti- 
vated by  new  medical  research  information  or  apparent  problems  that  have 
shown  up  frequently  in  the  review  of  exceptional  profiles.    Data  for  special 
studies  are  obtained  from  the  Reviewer  Request  program. 


PACE  REVIEW  PROCESS 

Review  Coordinator  Activities 

After  profiles  are  sent  to  UPRO,  they  are  read  by  the  review  coordinator 
who  identifies  obvious  errors  and  corrects  them.  Data  incorrectly  entered  will 
generate  history  file  transactions.  Corrections  are  then  entered  to  update  the 
Claims  History  File  and,  if  necessary,  the  Exception  Message  File.  To  expedite 
physician  review,  corrections  may  be  handwritten  and  forwarded  to  the  physician 
reviewers.  The  coordinator  also  applies  manual  review  guidelines  developed  and 
approved  by  the  PACE  Advisory  Ambulatory  Care  Review  Committee  or  the  Quality 
Evaluation  Committee. 


280 


Physician  Reviewer  Activities 

The  intent  of  physician  review  is  to  identify  and  ultimately  evaluate 
patterns  of  medical  care  at  variance  with  established  guidelines.  Each 
physician  reviewer  evaluates  patient  treatment  (according  to  the  patient 
and  provider  profiles)  in  comparison  with  established  criteria.    Each  excep- 
tion is  indicated  on  the  profile  by  a  flag  number  linking  the  exception  to  a 
message  defining  the  problem  source.    Exceptions  which  have  already  been 
evaluated  are  indicated  by  the  presence  of  the  review  disposition,  the  review 
date  and  the  reviewer's  initials.    Dispositions  are  identified  by  the  physician 
reviewer  marking  an  "X"  in  the  appropriate  disposition  category.  Dispositions 
are  then  added  to  the  History  Correct  Program,  which  updates  the  PACE  Suspense 
File,  Claims  History  File  and  Exception  Message  File  and  prints  each  review 
disposition  made  that  day  in  an  audit  report.    In  addition,  each  review  dis- 
position causes  a  record  to  be  added  to  the  Review  Disposition  File.    This  file 
is  then  added  to  the  MMIS  Suspense  File  causing  a  release  for  payment  or  denial. 

There  are  seven  disposition  options  for  the  MMIS-PACE  Module.  According 
to  the  documentation,  the  physician  reviewer  makes  not  only  a  medical  evalua- 
tion, but  also  executes  UPRO  policy  by  deciding  what  type  of  feedback  should 
be  given  to  the  physician  in  question  and  whether  any  payment  related  sanction 
is  needed.  By  choosing  a  particular  disposition,  the  physician  reviewer  auto- 
matically indicates  his  intended  feedback  action.  The  disposition  choices  for 
exceptions  are  as  follows: 

1.  Review  Unnecessary:    Any  flagged  medical  care  containing  this 
disposition  requires  no  further  review  and  this  flag  should  be 
deleted  from  patient  and  provider  histories. 

a.  Follow-up  action  by  reviewer:    Normally  all  exceptions  of 
these  types  will  be  cleared  during  preliminary  review  done 
by  Review  Coordinators.    However,  when  first  encountered 
by  physician  reviewers,  they  should  be  brought  to  staff's 
attention. 

b.  Follow-up  action  by  staff:    Log  in  any  coding  errors,  see 
they  are  corrected,  and  see  that  system  flaws  are  remedied. 

2.  Medically  Appropriate:    Although  the  care  did  not  match  screening 
parameters,  it  was  appropriate  given  the  circumstances  of  the 
case.    The  claim  is  cleared  for  payment. 

a.  Follow-up  action  by  reviewer:    Generally,  no  follow-up  is 
required.    However,  in  some  cases,  a  recommendation  for  re- 
evaluation  of  the  guideline  or  for  a  coding  change  may  be 
necessary. 

b.  Follow-up  action  by  staff:    Bring  any  problems  relating  to 
guidelines  to  appropriate  specialty  group  and  make  necessary 
coding  and  specification  changes.    Forward  these  to  OSI  for 
data  processing. 
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3.  Review  Inconclusive:    Information  available  was  not  sufficient 
to  make  a  conclusive  determination  about  the  appropriateness  of 
medical  care  involved.    If  a  new  claim  is  involved,  this  choice 
clears  it  for  payment. 

a.  Follow-up  action  by  reviewers:    Normally,  follow-up  action 

is  not  required,  nor  is  the  provider  notified  of  the  disposition. 

b.  Follow-up  action  by  staff:    Monitor  the  use  of  this  choice  for 
purposes  of  evaluating  the  PACE  concept  and  the  guidelines  in- 
volved and  identify  situations  which  require  physician  reviewer 
attention. 

4.  Not  Provider  Fault:    Medical  care  did  in  fact  vary  from  guidelines 
but  the  exception  appears  to  have  been  caused  by  factors  outside 
the  knowledge  or  control  of  the  flagged  provider.    This  choice 
will  also  clear  suspended  claims  for  payment. 

a.  Follow-up  action  by  reviewer:    Define  the  type  of  communication, 
if  any,  which  should  be  sent  to  the  provider  bringing  the  problem 
to  his  attention  and  asking  for  his  assistance  in  resolving  the 
situation.    Recommend  any  broad  educational  feedback  to  the 
general  professional  community  which  might  be  appropriate. 

b.  Follow-up  action  by  staff:    Prepare  necessary  correspondence  and 
file  a  response. 

5.  Inappropriate  Care  -  Education:    Exceptional  care  was  in  fact  not 
warranted  by  the  circumstances  of  the  case.    It  appears  that  an 
educational  effort  with  the  provider  would  be  productive  and  that 
no  payment  penalty  is  needed. 

a.  Follow-up  action  by  reviewer:    Make  a  direct  contact  with  the 
provider  or  request  that  a  letter  be  sent  to  explain  the  guide- 
line, its  rationale,  and  the  practice  which  deviated.    The  pro- 
vider's response  should  be  requested  and  where  appropriate, 
recommendations  made  for  general  educational  feedback. 

b.  Follow-up  action  by  staff:    Prepare  correspondence  and  file  any 
response. 

6.  Inappropriate  Care  -  Warning:    The  exceptional  care  in  question  is 
either  another  instance  of  an  unjustified  pattern  of  practice  or  by 
itself  represents  a  gross  departure  from  acceptable  medical  practice. 
This  code  will  still  clear  an  unpaid  claim  for  payment. 

a.    Follow-up  action  by  reviewer:    Make  a  direct  contact  with  the 
provider  or  request  that  a  letter  be  sent  to  inform  him  that 
future  claims  for  this  kind  of  care  cannot  be  medically  certi- 
fied for  reimbursement.    Again,  a  response  from  the  provider 
should  be  requested.    Define  parameters  for  a  temporary  screen 
to  suspend  any  subsequent  claim  of  the  same  type  by  that  pro- 
vider in  case  payment  penalties  must  be  imposed. 
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b.    Follow-up  action  by  staff:    Prepare  such  a  letter  and  file 
the  response.    Prepare  and  transmit  a  request  for  temporary 
screen.    Monitor  all  care  by  that  provider. 

7.    Inappropriate  Care  -  Provider  Fault  -  Denial:    Do  not  pay  this  claim. 
Note:    This  choice  can  only  be  recorded  for  claims  suspended  by  the 
State  (i.e.,  those  being  reviewed  on  a  prepayment  basis). 

a.  Follow-up  action  by  reviewer:    Request  a  letter  to  the  provider 
informing  him  of  UPRO's  action  and  methods  of  recourse  open  to 
him  and,  if  necessary,  modify  front-end  screening  parameters 
set  up  to  catch  new  claims  involving  similar  care. 

b.  Follow-up  action  by  staff:    Prepare  correspondence.  Transmit 
temporary  screening  parameters.    Monitor  all  care  by  that 
provider. 

Overseeing  the  physician  reviewers  is  the  responsibility  of  the  Ambulatory 
Care  Review  Committee.    If  a  provider  is  dissatisfied  with  the  decision  of  the 
review  physicians,  appeal  is  made  to  this  committee.    In  addition  to  this 
function,  the  committee  has  the  following  responsibilities: 

1.  The  committee  consists  of  the  following  primary  care  disciplines: 
family  practice,  internal  medicine,  obstetrics-gynecology ,  and  pedia- 
trics.   Representatives  from  other  disciplines  may  be  appointed  as 
necessary  by  the  PACE  Medical  Advisor. 

2.  The  PACE  Medical  Advisor  serves  as  chairman  of  the  committee. 

3.  Meetings  of  the  committee  are  normally  held  twice  a  month  at 
12:15  p.m.  on  the  second  and  fourth  Tuesdays  at  UPRO  headquarters. 

4.  In  accepting  an  appointment  to  the  Committee,  members  assume  the 
following  obligations  beyond  those  of  other  reviewers: 

a.  To  spend  at  least  an  hour  each  week  performing  review 

b.  To  attend  Committee  meetings  regularly 

c.  To  make  suggestions  whenever  possible  regarding  guide- 
lines, special  studies,  matters  to  be  communicated  to 
the  state,  subjects  for  general  educational  communica- 
tions and  other  agenda  items  for  the  Committee 

d.  To  sign,  as  needed,  letters  from  the  Committee  to  phy- 
sicians who  apparently  have  disregarded  previous  educa- 
tional correspondence 

e.  To  participate,  as  needed,  on  special  panels  to  prepare 
educational  materials,  to  hear  appeals,  etc. 

5.  Attendance  at  Committee  meetings  is  vital  to  the  success  and  cre- 
dibility of  the  PACE  program.    For  this  reason,  Committee  members 
and  guests  specifically  invited  to  present  information  on  agenda 
items  will  be  reimbursed  for  their  attendance.    All  reviewers  are 
welcome  at  Committee  meetings. 
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6.  Members  are  expected  to  notify  staff  if  they  will  not  be  able  to 
attend  and  should  solicit  the  attendance  of  another  reviewer 
from  their  specialty  in  their  place. 

7.  If,  for  some  reason,  a  member  finds  it  impossible  to  continue 
on  the  Committee,  he  should  notify  the  PACE  medical  advisor. 
Members  who  miss  three  consecutive  meetings  without  being  ex- 
cused will  be  replaced  by  the  PACE  medical  advisor  after 
appropriate  consultation. 


STRUCTURE  OF  THE  MMIS-PACE  MODULE 

The  requirements  for  this  module  have  been  developed  in  logical  groupings 
of  processing  functions  and  related  activities.    There  are  seven  groupings 
called  submodules.    Each  submodule  serves  as  a  point  of  entry  for  various  data 
elements  that  are  collected,  processed  and  displayed  on  one  or  more  outputs 
from  the  MMIS-PACE  Module.    They  are  as  follows: 

1.  Processing  Submodule 

2.  Provider  Subsystem  (MMIS) 

3.  Criteria  Evaluation  Submodule 

4.  Review  Feedback  Submodule 

5.  Reference  File  Subsystem  (MMIS) 

6.  Review  Request  Submodule 

7.  Maintenance  and  Update  Submodule 

They  are  described  as  follows: 

1.  Processing  Submodule 

The  processing  cycle  for  the  PACE  Module  begins  with  the  UPRO 
Review  File  generated  by  MMIS.    The  functions  involve  editing 
and  addition  of  claims  to  the  Claims  History  File  and  the 
maintenance  of  chain  points  for  providers  and  patients  in  the 
Provider  File  and  Patient  File. 

2.  Provider  Subsystem  (MMIS) 

The  Provider  File  is  used  by  the  PACE  Module  to  identify,  classify 
and  store  certain  statistical  information  pertinent  to  PACE  re- 
garding providers  in  the  Medicaid  program.    The  Provider  File  is 
consequently  used  in  editing  claims,  in  furnishing  backup  informa- 
tion used  in  the  medical  evaluation  process  and  in  identifying 
claim  and  exception  tallies  as  well  as  location  of  claims  in  his- 
tory for  individual  providers.    Although  the  PACE  Module  could  share 
the  Provider  File  with  MMIS,  operational  considerations  require  that 
PACE  use  a  copy  of  this  file.    The  MMIS  periodically  furnishes  a 
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tape  copy  of  the  Provider  File  to  the  PACE  Module  for  updating 
the  PACE  Provider  File. 

3.  Review  Feedback  Submodule 

This  submodule  provides  the  capability  to  modify  medical  care 
guidelines  and  history  data  in  the  PACE  Module  as  a  result  of 
review.    This  submodule  also  furnishes  the  feedback  linkage  to 
the  MMIS  Claims  Payment  System  with  a  pay  or  no-pay  disposition. 
The  three  functions,  claim  validation  and  correction,  physician 
review,  and  criteria  guideline  changes  are  also  incorporated  in 
this  submodule. 

4.  Reference  File  Subsystem  (MMIS) 

The  Reference  File  provides  the  PACE  Module  with  valid  Procedure, 
Drug  and  Diagnosis  information  for  use  in  the  claims  evaluation 
and  reporting  processes.    All  the  maintenance  of  the  Reference 
File  is  performed  by  the  MMIS.    Although  the  PACE  version  of  the 
Reference  File  is  a  copy  of  the  MMIS  Reference  File,  the  PACE 
version  does  not  reflect  deletions  made  to  the  MMIS  Reference 
File,  since  older  claims  stored  in  the  PACE  History  File  may 
contain  diagnosis,  procedure  and  drug  codes  which  have  been 
deleted  from  the  active  MMIS  files. 

5.  Review  Request  Submodule 

This  submodule  is  designed  to  allow  flexibility  in  the  retrieval 
of  historical  information  on  file.    The  submodule  can  present 
information  in  several  ways  on  an  "as  needed"  basis.    They  are 
profile  requests,  summary  reports,  etc.  -  as  discussed  previously. 

6.  Maintenance  and  Update  Submodule 

This  submodule  contains  those  functions  which  are  a  necessary 
component  of  any  data  processing  system,  although  not  considered 
an  integral  part  of  the  normal  processing  cycle.  It  provides 
backup  and  recovery  procedures,  file  reorganization,  Patient, 
Master  and  Patient  Cross-Reference  File  maintenance  capability, 
and  a  means  of  printing  the  contents  of  various  files  used  by 
the  PACE  Module. 


OPERATIONAL  STATUS 

To  partially  assess  the  operational  status  of  the  UPRO-PACE  system, 

investigators  reviewed  and  abstracted  minutes  from  the  Ambulatory  Review 

Committee.    The  minutes  of  these  meetings  covered  nine  months  and  15  meetings. 
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The  minutes  do  not  necessarily  describe  specific  operational  aspects  of  the 
PACE  computer  system,  but  do  describe  the  activities  of  a  professional  review 
committee  using  PACE  generated  information  as  a  basis  for  actions. 

While  the  UPRO-PACE  system  is  concerned  with  professional  review  of 
Medicaid  care  in  Utah,  equally  important  aspects  of  PACE  operations  deal  with 
research  of  an  ambulatory  review  method  and  the  application  of  an  automated 
system  to  such  review.    The  previous  section  describes  the  PACE  method  and 
functions,  but  it  is  important  to  note  the  results  of  such  method  and  auto- 
mated processing  for  ambulatory  review. 

Generally,  the  PACE  review  system  is  operating  as  outlined  in  the  pre- 
vious section;  however,  because  the  system  is  complex,  constant  revision  is 
conducted  by  UPRO  personnel  making  a  specific  operational  status  difficult  to 
describe.    During  the  major  system  revision  for  MMIS-PACE  integration  (July 
1974  to  September  1975),  both  UPRO  and  OSI  personnel  invested  substantial 
resources.    The  purpose  of  the  redesign  for  MMIS-PACE  integration  resulted 
in  increased  software  capability  and  expanded  criteria  items.    This  section 
will  describe  the  results  of  that  investment  in  terms  of  actions  taken  by 
UPRO  based  on  PACE  generated  data  and  assessment. 

The  Ambulatory  Care  Review  Committee  (ACRC)  minutes  are  outlined  accor- 
ding to  specific  action  items  and  discussion  topics  within  each  meeting. 
Action  items  contained  in  the  minutes  should  provide  an  indication  of  the  type 
and  level  of  committee  review  activity;  however,  this  is  not  to  say  all  UPRO 
operational  activities  were  documented  in  ACRC  minutes.    The  contents  of 
minutes  are  outlined  below  beginning  with  the  earliest  meetings.    In  terms  of 
attendance,  the  usual  pattern  included  the  chairman,  several  review  physicians, 
several  PACE  staff  members  and  some  guests.    Meetings  usually  last  an  hour  and 
a  half. 


Meeting  #1    (October,  1975) 

a.  Two  medical  guidelines  developed 

b.  Discussion  of  guidelines  to  govern  sending  warning  letters 
to  providers 

Meeting  #2    (November,  1975) 

a.    Discussion  of  policy  for  transmitting  drug  usage  information 
to  state  medicaid  personnel 

Meeting  #3    (December,  1975) 

a.    One  medical  guideline  developed 

Meeting  #4    (December,  1975) 

a.  Three  warning  letters  sent  to  provider  regarding  three  different 
drug  usage  cases,  because  patients  received  same  treatment  after 
an  initial  educational  letter  was  sent  to  provider 
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Meeting  #5    (January,  1976) 


a.  Discussion  of  three  drug  usage  cases  to  be  presented  to  state 

b.  Approval  of  educational  correspondence  format 

c.  Three  warning  letters  approved  to  be  sent  out  to  providers 

d.  One  medical  guideline  recommended  for  input  to  PACE 

Meeting  #6    (January,  1976) 

a.  Discussion  of  Committee's  investigative  approach  for  excessive 
drug  usage 

b.  Discussion  of  different  drug  criteria 

c.  One  new  medical  guideline  recommended  for  input  to  PACE 

Meeting  #7    (February,  1976) 

a.  Five  problem  provider  profiles  and  committee  physician  telephone 
contacts  with  those  providers  discussed.    Three  referred  to  state 
and  two  to  be  recontacted  by  committee 

b.  Four  medical  guideline  revisions  discussed 


Meeting  #8    (March,  1976) 

a.  Three  drug  abuse  cases  and  committee  physician  telephone  contacts 
with  case  providers  discussed.    All  three  cases  referred  to  state 

b.  Four  medical  guideline  revisions  discussed 

c.  General  discussion  of  committee  follow-up  procedures  to  providers 
identified  in  review  process 

d.  Committee  reviewed  frequencies  of  diagnosis  and  procedures  for 
Medicaid  population 


Meeting  #9    (March,  1976) 

a.  Three  drug  usage  cases  discussed,  committee  physicians  assigned  to 
contact  providers 

b.  Three  drug  usage  follow-ups  discussed,  including  discussion  of 
committee  physicians  telephoning  case  providers.    Two  cases 
referred  to  state,  one  to  be  followed  up  by  ACRC  chairman 

c.  Two  medical  guideline  revisions  discussed 

Meeting  #10    (April,  1976) 

a.  Four  drug  usage  follow-ups  discussed,  including  discussion  of 
committee  physicians  telephoning  case  provider.    All  four  cases 
referred  to  state  Medicaid  personnel 

b.  Five  medical  guideline  revisions  discussed 
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Meeting  #11    (April,  1976) 


a.  Four  drug  usage  cases  discussed,  three  referred  to  state  Medicaid 
personnel  for  action 

b.  Five  drug  usage  case  follow-ups  by  state  Medicaid  personnel 
discussed,  committee  physicians  further  discussing  ways  to 
improve  the  State's  resolution  of  such  problem  cases 

c.  Five  medical  guidelines  revisions  discussed 

Meeting  #12    (May,  1976) 

a.  Three  drug  usage  cases  discussed 

b.  One  "physician  shopper"  case  was  discussed 

c.  Six  drug  usage  follow-ups  were  discussed,  including  assigning  committee 
physicians  to  telephone  these  providers  and  discuss  case  characteristics 

d.  Three  medical  guideline  revisions  were  discussed 

Meeting  #13    (May,  1976) 

a.  Four  drug  usage  cases  were  discussed,  although  no  action  was  noted 

b.  Three  medical  guidelines  revisions  discussed,  and  changes  made 

c.  Two  educational  correspondences  sent  out  and  a  discussion  of  one 
physician  response  was  noted 

Meeting  #14    (June,  1976) 

a.  Two  drug  usage  cases  discussed,  although  no  action  was  noted 

b.  Four  medical  guideline  revisions  discussed  and  changes  made 

c.  General  discussion  of  educational  correspondence  to  physicians 
with  exceptions 

Based  on  the  minutes,  the  level  of  activity  attributed  to  PACE  seems  high. 
Minutes  do  not  cover  the  feedback  and  action  activities  of  all  review  physicians, 
although  some  of  that  activity  is  noted.    Review  physicians  are  responsible  for 
telephoning  providers  who  have  been  identified  through  the  review  process  as 
being  a  recurring  problem  in  one  area  (both  drug  usage  and  prescription  patterns). 

As  documented  above,  the  committee  activities  centered  on  drug  usage.  The 
interaction  between  the  state  and  UPRO  seems  critical,  since  after  educational 
telephone  calls,  UPRO  refers  most  cases  to  the  state. 

The  minutes  reflect  varying  levels  of  detail.    The  earlier  minutes  were 
less  detailed  than  the  later  minutes,  perhaps  reflecting  the  increased  activity 
of  the  entire  review  system.    The  minutes  only  reflect  these  activities  of  the 
PACE,  although  some  review  actions  could  have  been  conducted  by  different 
organizational  components  of  UPRO. 
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SECTION  III 


HCMS     STUDY  INFORMATION 


The  Utah  Professional  Review  Organization  was  included  in  the  study 
because  it  is  one  of  two  existing  PSROs  conducting  ambulatory  quality 
review.    The  documentation  used  to  provide  an  overview  of  the  system  and 
to  substantiate  many  points  discussed  during  the  interviews  is  as  follows: 

1.  Listing  of  the  PACE  Physician  Reviewers 

2.  Selection  of  Evaluation  Choices  and  Notes  on  Review  Evaluation 
Choices  (April  16,  1976) 

3.  Flow  chart  of  the  Regular  Pace  Processing  Cycle  (May  26,  1976) 

4.  Grant  application  to  the  Department  of  Social  Services  for  the 
implementation  of  the  Physician  Ambulatory  Care  Evaluation 
Module  Program  (April  29,  1975) 

5.  Utah  Medicaid  PACE  Module  (June  16,  1975)  presented  to  UPRO  by 
OS I  containing: 

Volume  I  -  Requirements  Analyses 
Volume  II  -  Design  Considerations 
Volume  III  -  Module  Description 
Volume  IV  -  Schedule  of  tasks 

6.  Memo  to  file  from  James  Cannon  about  the  Frequency  Listing  of 
Diagnoses,  Procedures  and  Drugs  (March  1,  1976) 

7.  List  of  Specialty  Codes  and  Abbreviations 

8.  Letter  addressed  to  E.  Woehrmann  from  OSI  about  the  hardware 
description  and  cost  (January  23,  1976) 

9.  Responsibilities  of  PACE  Advisory  Committee  (February  10,  1976) 

10.  Letter  and  article  sent  to  physicians  from  the  Academy  for 
Continuing  Education  regarding  the  use  of  injectable  penicillin 
(October  15,  1975) 

11.  UPRO-PACE  Guidelines  Description  Report  (March  16,  1976) 

12.  Paper  written  by  E.  Woehrmann  documenting  flow  of  claims  data 
in  MMIS  and  PACE  system  (April  19,  1976) 

13.  Various  output  forms 

14.  Health  Care  Management  Systems,  Inc.,  Evaluation  Report: 
Physician  Ambulatory  Care  Evaluation  Module  Volume  Four: 
Interim  Cost  Study  -  1975,  March,  1976 

HCMS  staff  visited  UPRO  on  May  27  and  June  1  of  1976  and  interviewed  the 
following  personnel : 

1.  Executive  Administrator 

2.  Director  of  Operations 

3.  PACE  Project  Manager 
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4.  Medical  Advisor  to  PACE  Advisory  Ambulatory  Care  Review  Committee 

5.  Review  Coordinator 

The  following  topics  were  covered  during  the  interviewed: 

1.  Funding  mechanisms  for  PACE 

2.  Development  of  PACE  via  OSI;  UPRO  interaction 

3.  Relationship  between  PACE,  State  Medicaid  Program  and  MMIS 

4.  Responsibilities  of  staff  with  regard  to  PACE  program 

5.  Number  of  claims  filed  monthly 

6.  Quality  Evaluation  Committee  functions  with  regard  to  the 
establishment  and  revision  of  criteria 

7.  Screening  process 

8.  Physician  Reviewers'  and  PACE  Advisory  Ambulatory  Care  Review 
Committee  functions  with  regard  to  review  process  and 
decision-making 

9.  Reports  generated  through  PACE 

10.  Disposition  of  claims 

11.  Educational  feedback  mechanism 

12.  Recent  refinements  in  PACE 

13.  Feasibility  of  UPRO-PACE  in  other  settings 

14.  Future  PACE  operation 


290 


NEW  MEXICO  PROFESSIONAL  STANDARDS  REVIEW  ORGANIZATION 
Albuquerque,  New  Mexico 


SECTION  I 
INTRODUCTION 


The  New  Mexico  Professional  Standard  Review  Organization  (NMPSRO)  was 
officially  established  and  incorporated  in  1973,  having  evolved  from  the  New 
Mexico  Foundation  for  Medical  Care  (NMFMC)  established  in  April,  1970.  The 
NMFMC  grew  out  of  efforts  by  the  New  Mexico  Medical  Society  to  explore  the 
possibility  of  organizing  a  medical  care  foundation  in  New  Mexico.    The  impetus 
for  development  originated  with  a  group  of  Medical  Society  physicians  interested 
in  examining  a  formal  approach  to  monitoring  medical  care  activities. 

During  this  time,  the  New  Mexico  Medicaid  program  was  undergoing  a  fiscal 
crisis  involving  rising  Medicaid  costs.    There  was  pressure  on  the  state  Medicaid 
agency  and  the  Health  and  Social  Services  Department  (HSSD)  to  control  costs. 
The  state's  Medicaid  fiscal  agent  was  apparently  unable  to  adequately  control 
and  project  the  costs. 

Soon  after  NMFMC 's  establishment,  HSSD  expressed  an  interest  both  in  em- 
ploying a  new  fiscal  agent  and  in  contracting  with  the  NMFMC  to  provide  pro- 
fessional review  of  Medicaid  cases.    NMFMC  physicians  were  not  passive  in  these 
developments;  they  were  also  dissatisfied  with  the  Medicaid  program  and  welcomed 
the  opportunity  to  perform  professional  review.    Once  a  new  fiscal  agent  was 
chosen,  the  state  wrote  independent  contracts  with  NMFMC  for  professional  review 
and  with  the  agent  for  claims  processing  and  payment. 

A  second  significant  development  was  the  award  of  an  Experimental  Medical 
Care  Review  Organization  (EMCRO)  grant  to  NMFMC  in  March,  1972.    The  EMCRO  goals 
were  incorporated  into  the  NMFMC  overall  plan  to  conduct  professional  review, 
but  required  NMFMC  to  conduct  certain  research  and  development  evaluation  acti- 
vities.    Research  and  development  focused  on  establishing  and  testing  (a)  an 
automated  institutional  claims  review  system  and  (b)  alternatives  for  performing 
professional  review.     Evaluation  activities  included  examining  various  prac- 
tice and  utilization  patterns  for  Medicaid  providers  and  recipients.    The  EMCRO 
funding  continued  through  February,  1975. 

The  passage  of  PSRO  legislation  (P.L.  92-603)  precipitated  the  development 
of  the  NMPSRO  as  a  distinct  organizational  unit.    This  organization  absorbed 
the  professional  review  functions  of  NMFMC.    In  April,  1974,  NMPSRO  applied  for 
designation  as  New  Mexico's  conditional  PSRO,  with  the  NMFMC  personnel  playing 
a  concurrent  role  in  the  PSRO's  development.    Before  receiving  conditional  sta- 
tus in  November,  1974,  NMPSRO  was  required  to  go  through  the  polling  process 
since  more  than  10%  of  the  licensed  physicians  (12%)  objected  to  the  proposed 
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designation.    The  results  of  the  poll  were  that  73%  of  those  voting  approved 
the  designation. 

The  following  section  describes  the  chain  of  events  in  development  of  the 
present  ambulatory  review  system. 
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SECTION  II 

QUALITY     ASSURANCE  PROGRAM 


BACKGROUND 

In  August,  1975,  the  New  Mexico  PSRO  was  granted  $441,000  from  the  Social 
Rehabilitation  Services  (SRS),  Department  of  Health,  Education  and  Welfare,  to 
implement  a  claims-based  ambulatory  review  system  for  Title  XIX  provider  and 
patient  populations. 

Prior  to  development  of  this  system,  professional  review  of  Title  XIX 
services  was  performed  by  the  New  Mexico  Foundation  for  Medical  Care  (1971). 
This  original  review  scheme  was  taken  over  by  the  New  Mexico  PSRO  (November, 
1974),  which  conducted  prepayment  review  of  all  Medicaid  claims  submitted  for 
payment.    At  this  time,  one  multipurpose  claim  form  was  used,  requiring  providers 
to  write  in  diagnostic  and  therapeutic  information.    Review  entailed  a  manual, 
clerical  audit  of  each  claim  by  comparing  claim  data  with  explicit  guidelines 
developed  by  NMPSRO  physicians.    The  guidelines  defined  ranges  of  procedures 
(services,  treatments)  for  specific  diagnoses.    When  the  clerical  examiner  dis- 
covered claims  which  violated  the  guidelines,  the  claims  were  referred  to 
physicians  for  professional  review.    Professional  review  decisions  fell  into 
five  categories:    (a)  return  the  claim  for  further  information,  (b)  refer  to 
another  reviewing  physician,  (c)  refer  to  a  panel  of  physicians,  (d)  deny  pay- 
ment and  (e)  grant  payment. 

Throughout  NMPSRO' s  implementation  of  a  100%  review  system,  questions  were 
raised  regarding  the  cost  effectiveness  of  conducting  total  claims  review.  In 
late  1974,  the  NMPSRO  Research  and  Development  Committee  began  to  study  the 
system's  effectiveness.    This  analysis  focused  on  the  massive  amounts  of  data 
collected  during  the  EMCR0  program  and  possible  redesign  of  the  100%  claims 
review  system. 

NMPSRO  personnel  indicated  that  the  Bureau  of  Quality  Assurance  (BQA), 
Department  of  Health,  Education  and  Welfare,  informed  the  State  of  New  Mexico 
that  a  more  efficient  and  less  costly  review  system  had  to  be  implemented  for 
additional  federal  funding  to  be  appropriated  for  review.    Both  NMPSRO  and  Dike- 
wood  Industries  (Title  XIX  fiscal  agent  for  the  state)  were  responsible  for  system 
redesign,  with  assistance  from  several  external  organizations  and  individuals. 
Between  March  and  May,  1975,  it  became  evident  to  NMPSRO  and  Dikewood  personnel 
that  such  a  system  redesign  should  take  place  on  a  gradual  basis  over  a  twelve- 
month period.    Funding  limitations,  however,  forced  system  revision  to  take 
place  more  quickly  in  July,  1975,  constraining  the  time  for  detailed  preparation. 

The  first  major  system  revision  involved  the  development  of  precoded  claim 
forms  to  be  used  by  providers.    As  noted  above,  the  New  Mexico  Medicaid  system 
had  been  using  one  multipurpose  form.    There  were  several  distinct  reasons 
noted  for  moving  to  a  precoded  form:    (a)  HSSD  wanted  to  reduce  the  time  in 
claims  processing  and  payment,  hopefully  to  make  it  more  cost-efficient,  (b) 
a  more  accurate  method  for  capturing  claims  data  was  needed,  and  (c)  it  was 
felt  that  reducing  the  amount  of  manual  coding  by  NMPSRO  personnel  might  reduce 
coding  errors.    Because  the  old  Medicaid  forms  required  providers  to  write  in 
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claims  information,  NMPSRO  coders  often  had  difficulty  interpreting  the  problems 
or  diagnoses  and  procedures  indicated  by  providers.    Due  to  NMPSRO's  position 
that  manual  coding  should  be  handled  by  the  professional  review  organization, 
HSSD  had  originally  contracted  with  NMPSRO  to  do  all  coding  of  claims  informa- 
tion (rather  than  with  Dikewood,  the  Medicaid  fiscal  agent). 

The  new  precoded  form  was  developed  principally  by  NMPSRO  personnel,  with 
assistance  from  HSSD  and  Dikewood  personnel.    After  initial  development  of  the 
form,  NMPSRO  pretested  the  pediatrics  form  on  four  physicians.    NMPSRO  reported 
the  pretest  was  generally  successful.    The  one  significant  problem  in  the  develop- 
ment of  the  form  involved  printing  the  form  accurately.    Since  HSSD  was  responsibl 
for  printing  the  form,  a  state  printing  contractor  was  required  to  do  the  printing 
The  printer  was  located  in  another  state,  leading  to  significant  difficulties  and 
delays  in  completing  a  final  draft  of  the  form. 

Once  the  new  precoded  form  was  finalized  (August,  1975),  NMPSRO  identified 
a  portion  of  the  physician  population  to  begin  a  phased  implementation  program. 
NMPSRO  indicated  133  physicians  are  currently  using  the  precoded  forms,  and 
75  to  80%  of  the  forms  are  going  through  the  system  without  major  processing 
problems.    NMPSRO  and  Dikewood  personnel  developed  an  information  booklet  for 
providers  to  follow,  and  conducted  some  training  in  physician  offices.  NMPSRO 
personnel  indicated  there  had  been  problems  distributing  new  forms  to  the  right 
providers,  and  that  some  of  the  high  utilization  providers  had  not  yet  received 
forms . 

The  second  major  revision  dealt  with  eliminating  100%  review  of  all  Medicaid 
claims.    Two  types  of  review  are  now  occurring: 

1.  Focused  prepayment  review  of  approximately  10%  of  the  Medicaid  provider 
population 

2.  Retrospective  review,  identifying  providers  with  ten  or  more  encounters 
during  a  six-month  period  for  a  single  diagnosis 

The  prepayment  review  mechanism  identifies  providers  who  have  previously 
had  a  high  rate  of  claims  denial.    After  the  system  was  in  operation,  the  retro- 
spective system  component  identified  providers  to  be  reviewed  before  payment. 
During  the  100 %  prepayment  review,  approximately  60  physicians  had  a  high  claim 
denial  rate.    These  physicians  made  up  the  review  population  (approximately  10%) 
for  the  new  prepayment  review.    All  claims  in  this  group  are  reviewed  by  the 
review  coordinator  and  then  by  one  of  the  review  physicians.    The  specific  pro- 
cedural steps,  outlined  by  NMPSRO  personnel,  are: 

1.  A  clerical  examiner  reviews  the  claims  against  processing  and  eligi- 
bility guidelines  referring  cases  which  fall  outside  the  quidelines 
to  professional  review. 

2.  The  review  coordinator  reviews  each  claim  to  ascertain  whether  any 
problems  lie  in  the  provider's  service. 

3.  If  a  problem  is  cited,  the  review  coordinator  catalogs  the  problems 
by  specialty  groups.    The  review  coordinator  uses  a  CRT  terminal  to 
request  orovider  history  to  accompany  the  data  being  consulted  by 
the  review  physician  during  his  assessment. 

4.  The  review  coordinator  schedules  sessions  for  NMPSRO  review  physicians. 
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5.  The  review  coordinator  accompanies  the  physician  reviewers  while 
review  is  conducted  (to  insure  that  the  physician  documents  rea- 
soning for  his  or  her  medical  judgment). 

6.  If  review  reveals  inappropriate  care,  the  physician  is  denied  payment 
for  the  services  in  question. 

7.  Unusual  or  outstanding  cases,  as  well  as  appeals  of  payment  denial, 
are  brought  before  the  Ambulatory  Care  Review  Committee  as  to 
appropriate  action. 

One  of  the  most  important  functions  of  prepayment  review  is  to  allow 
denial  of  payment  for  overutilization  of  services.    Denial  is  decided  upon  at 
the  physician  reviewer  level. 

All  Title  XIX  providers  providing  care  to  eligible  recipients  are  subject 
to  retrospective  review.    Providers  identified  through  retrospective  review  as 
high  utilizers  may  be  placed  on  prepayment  review.    Retrospective  review  begins 
with  the  review  coordinator  examining  a  list  of  diagnoses  with  the  highest  fre- 
quencies.   This  information  was  based  on  four  years  of  data  collected  through 
the  EMCRO  program.    More  specifically,  a  diagnosis  frequency  report  from  the 
EMCRO  ambulatory  care  data  tapes  is  perused  by  the  review  coordinator.  Once 
the  review  coordinator  identifies  the  diagnosis  topic  to  be  reviewed,  she  re- 
quests recipient  and  provider  profile  information  from  Dikewood  that  lists  all 
physicians  who  have  had  10  or  more  patient  encounters  for  the  chosen  diagnosis 
during  a  six-month  period.    Retrospective  review  uses  three  reports,  called 
Level  I,  Level  II  and  Level  III  reports.    Once  the  review  coordinator  chooses 
a  diagnosis  (or  similar  diagnostic  group)  for  review,  she  will  request  the 
Level  I  report.    This  report  divides  providers  into  the  10th,  50th  and  90th 
percentile  groups  and  has  ten  columns  for  data  reporting.    The  first  three 
columns  are  fixed,  and  note  the  following  information  for  each  diagnosis: 

Column  1  -  Physician  identification  number 

Column  2  -  Number  of  patients  encountered  by  the  provider  for  the  identi- 
fied diagnosis 

Column  3  -  The  number  of  encounters  per  patient  for  the  identified  diag- 
nosis 

The  tenth  column  is  also  fixed,  and  notes  the  cost  per  patient  for  that 
specific  diagnosis,  based  on  information  in  the  other  columns.    Columns  four 
through  nine  are  variable  and  any  element  or  combination  of  elements  of  care 
may  be  obtained  as  long  as  they  are  captured  in  the  New  Mexico  Medicaid  Manage- 
ment Information  System,  including: 

1 .  Laboratory  tests 

2.  Injections 

3.  Prescriptions 

4.  X-rays 

5.  Outpatient  surgical  procedures 

6.  Electrocardiograms,  etc. 

The  variable  columns  allow  the  review  coordinator  to  request  several  Level  I 
reports  for  any  given  diagnosis.    For  initial  Level  I  analysis  in  the  retrospec- 
tive system,  the  review  coordinator  requests  a  "standard  default"  format. 
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This  format  includes  five  basic  areas  in  the  variable  columns  which  the  review 
coordinator  uses  to  identify  those  providers  who  require  more  focused  Level  I 
reports,  as  well  as  Level  II  and  III  reports  for  more  indepth  review.    The  five 
"standard  default"  format  elements  are: 

1.  Laboratory  tests  per  patient 

2.  X-rays  per  patient 

3.  Prescriptions  per  patient 

4.  Injections  per  patient 

5.  All  others  per  patient 

The  review  coordinator  also  consults  physician  review  specialists  and  uses 
medical  textbooks  to  determine  therapy  and  diagnostic  elements  that  are  speci- 
fically contraindicated.    These  elements  all  become  columns  in  as  many  Level  I 
reports  as  are  required  to  examine  the  diagnosis  completely.    In  this  way, 
specific  treatment  patterns  that  might  not  otherwise  appear  can  be  examined. 
Documents  note  that  the  coordinator  uses  the  New  Mexico  Criteria  for  Outpatient 
Care,  in  consultation  with  reviewing  physicians,  when  requesting  different  com- 
binations of  categories  (e.g.,  different  types  of  lab  tests,  different  injections, 
etc.).    For  example,  she  could  request  viewing  six  specific  types  of  injections 
for  a  specific  diagnosis  in  the  variable  columns.    The  analysis  of  that  request 
would  then  be  based  on  those  injections  compared  with  the  diagnosis,  not  including 
laboratory  tests  or  prescriptions. 

All  Level  I  reports  are  divided  into  three  statistical  distributions,  10th, 
50th  and  90th  percentiles.    The  review  coordinator  will  focus  analysis  on  the 
two  extreme  percentiles  for  each  category  of  service.    Level  I  reports  are 
reported  by  different  sorting  processes.    For  example,  if  the  review  coordinator 
wanted  Level  I  sorted  for  prescriptions,  the  column  with  prescriptions  per 
patient  would  be  ranked  from  highest  to  lowest  users  (90th,  50th,  and  10th  per- 
centiles).   The  review  coordinator  could  then  examine  this  column  and  cite  over 
or  underutil ization.    The  review  coordinator  is  supplied  with  printouts  showing 
all  the  columns  sorted  from  highest  to  lowest  in  utilization. 

After  the  review  coordinator  reviews  the  Level  I  reports  and  cites  physi- 
cians, she  orders  Level  II  and  III  reports  for  further  review  of  the  cases  to 
determine  the  validity  of  the  citing.    When  examining  Level  I,  the  review  coor- 
dinator employs  a  cite  sheet.    The  cite  sheet  has  several  informational  points: 
(a)  dates  of  review,  (b)  diagnosis  order  review,  (c)  cited  physician's  identifi- 
cation number  and  (d)  indication  for  high  or  low  utilization.    There  are  seven 
columns  in  which  the  review  coordinator  can  indicate  the  citation  for  the  physi- 
cian (e.g.,  injection,  laboratory  tests,  etc.). 

Level  II  reports,  by  physician  and  primary  diagnosis,  detail  an  individual 
physician's  performance  along  six  elements  of  care,  comparing  the  performance 
of  a  "cited"  physician  with  computed  medians  for  the  rest  of  the  Medicaid  provider 
population  who  had  at  least  ten  patient  encounters  in  the  period  of  review.  Level 
II  reports  are  divided  into  four  sections: 
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Section  1  gives  an  introduction  and  explanatory  comments  for  the  following 
"  sections 


Section  2  details  the  physician's  performance,  over  the  six-month  period 
of  review,  comparing  it  with  the  median  of  the  provider  popula- 
tion for  nine  elements: 


(a 
(b 
(c 
(d 
(e 
(f 
(9 
(h 
(i 


Number  of  patients 
Number  of  encounters 
Encounters  per  patient 
Number  of  services 
Services  per  patient 
Services  per  encounter 
Number  of  dollars  paid 
Dollars  paid  per  patient 
Dollars  paid  per  encounter 


Section  3  provides  a  matrix  for  comparing  the  physician  with  the  provider 
population  for  number  per  patient,  number  per  encounter  and  per- 
cent of  patients  for  the  following  elements  of  care: 

(a)  Visits 

(b)  Laboratory  tests 

(c)  X-rays 

(d)  Prescription  drugs 

(e)  Injections 

(f)  Other  procedures 


Section  4  provides  the  same  comparison  as  in  Section  3,  but  specifies  for 
the  seven  elements  of  care  up  to  five  individual  procedures, 
always  having  as  a  final  element  "all  other"  procedures. 

The  review  coordinator  examines  Level  II  reports  and  circles  or  checkmarks 
the  physician  population  medians  where  problems  may  exist. 

After  reviewing  this  report,  the  review  coordinator  will  then  review  Level 
III  reports.    Level  III  reports  list  by  diagnosis  every  encounter  in  the  six- 
month  review  period  for  a  physician  under  review.    This  report  details  seven 
data  elements: 

1 .  Patient  number 

2.  Sex 

3.  Age 

4.  Date  of  service 

5.  Secondary  diagnosis 

6.  All  outpatient  procedures 

7.  Prescription  drugs 

Based  on  the  analysis  of  Level  II  and  III  reports,  the  review  coordinator  will 
decide  whether  to  forward  the  case  to  the  physician  reviewers.    If  the  provider1 
treatment  pattern  is  comparable  to  the  population  median  and  criteria  are  not 
violated,  physician  review  is  not  necessary.    If  any  problem  is  identified, 
the  case  is  scheduled  for  medical  review.    This  decision  to  forward  cases  for 
physician  review  is  left  to  the  review  coordinator,  and  there  is  currently  no 
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systematic  format  for  reaching  this  decision.    NMPSRO  personnel  indicated  that 
since  the  provider  population  is  relatively  small  and  the  variance  situation 
fairly  clear-cut,  the  review  coordinator  has  the  ability  and  experience  to  make 
such  a  decision. 

Once  the  review  coordinator  decides  physician  review  is  necessary,  she 
attaches  a  form  to  the  reports  which  includes  space  for  the  following  information 

1.  Number  of  physician  being  reviewed 

2.  Specialty 

3.  Address 

4.  Elements  of  care  reviewed 

5.  Review  coordinator  comments 

6.  Physician  reviewer  comments 

7.  Physician  reviewer  decision  -  six  separate  actions  are  possible 

8.  Physician  reviewer  signature 

9.  Date  of  review 

This  form  essentially  documents  the  review  process.    The  review  coordinator 
contacts  the  reviewing  physician,  schedules  a  session  and  attends  the  session 
with  the  physician.    The  reviewing  physician  has  six  choices  of  action: 

1.  Forward  case  to  physician  for  prepayment  review 

2.  Refer  case  to  Ambulatory  Review  Committee 

3.  Refer  to  another  reviewer 

4.  Contact  reviewed  physician  for  more  information 

5.  No  action 

6.  Refer  case  to  Education  Committee 

The  review  coordinator  must  attend  review  sessions  in  order  to  insure  that  the 
reviewing  physician  documents  the  reasons  for  his  decision.    NMPSRO  personnel 
indicated  that  the  medical  guidelines  and  PSRO  policy  developed  for  the  specific 
diagnosis  or  for  certain  procedures  should  aid  the  reviewer's  decision,  since 
the  initial  variance  decision  was  based  on  those  guidelines. 

After  completion  of  physician  review,  the  two  professional  committees  may 
begin  their  role  in  the  review  process.    A  reviewer  can  make  a  decision  to  for- 
ward a  case  to  the  Education  Review  Committee,  the  Ambulatory  Care  Review 
Committee,  or  both.    Essentially,  a  reviewer  determines  whether  the  nature  of 
the  problem  is  an  educational  matter  or  concerns  utilization.    The  education 
issues  will  normally  be  forwarded  to  the  Education  Committee,  which  takes 
appropriate  action.    Utilization  issues  are  forwarded  to  the  Ambulatory  Care 
Review  Committee,  which  implements  corrective  action.    There  have  been  instances 
in  which  both  committees  have  taken  simultaneous  action  on  a  case,  or  one  com- 
mittee has  referred  a  case  to  the  other  committee.    The  review  coordinator, 
who  monitors  the  status  of  cases  going  through  the  system,  has  several  data 
flow  sheets  which  document  the  status  of  each  case.    The  review  coordinator 
attends  both  education  and  ambulatory  review  meetings. 
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Education  Committee 


Documents  note  the  Education  Committee  is  composed  of  eleven  reviewing 
physicians.    Once  the  committee  receives  a  case  forwarded  by  a  review  physician 
for  educational  purposes,  the  committee  must  initially  decide  whether  education 
is  appropriate.    If  the  committee  decides  an  education  program  should  be  imple- 
mented, they  must  then  decide  what  type  of  education  is  necessary.    There  are 
several  options: 

1 .  Send  a  letter 

2.  Send  a  letter  with  literature  attached 

3.  Ask  the  physician  to  come  for  an  educational  exchange  session  with 
NMPSRO  physicians 

In  terms  of  documenting  operational  status  of  the  system,  HCMS  study  staff 
abstracted  information  from  a  recent  NMPSRO  status  report  and  substantiated 
the  report  by  reviewing  all  Education  Review  Committee  minutes. 

The  Status  Report  of  the  Ambulatory  Review  System  (February  3  to  May  6, 
1976)  presented  the  following  information: 

Education  Committee  Members  Hours  -  28-1/4 
Letters  to  Providers  -  27 

Referrals  to  Education  Committee,  but  no  action  taken  -  19 

The  Status  Report  also  indicated  each  committee  held  four  meetings  between 
February  and  May,  with  good  attendance.    There  were  three  physician  replies  to 
Education  Committee  actions.    In  each  case,  the  committee  responded  to  the  reply, 
and  they  monitored  each  case. 


Ambulatory  Care  Review  Committee 

The  Ambulatory  Care  Review  Committee  (ACRC)  is  composed  of  twelve  physi- 
cians.   The  basic  function  of  the  committee  is  to  review  cases  referred  by 
physician  reviewers  in  which  utilization  issues  seem  paramounts  and  to 
decide  whether  a  physician  should  be  placed  on  prepayment  review.    If  a 
physician  is  placed  on  prepayment  review,  the  ACRC  chairman  sends  a  letter 
to  the  physician  indicating  the  reasons  for  which  he  has  been  placed  on 
review  and  the  effects  of  such  review  on  claims  processing.    ACRC  also  governs 
and  monitors  reviewing  physicians.    For  example,  if  a  reviewer  seems  to  be 
referring  cases  to  committee  unnecessarily  (or  unnecessarily  denying  payments 
in  prepayment  review),  ACRC  will  review  his  cases  and  take  appropriate  action. 

When  the  review  system  bacame  operational  in  February,  1976,  60  physicians 
were  on  prepayment  review.    Since  that  time,  the  Ambulatory  Review  Committee 
has  placed  an  additional  four  physicians  on  prepayment  review. 


Title  XIX  Claims  Processing 

As  indicated,  the  New  Mexico  Medicaid  system  is  processing  both  precoded 
and  nonprecoded  claims.    Nonprecoded  claims  enter  the  system  at  Dikewood  directly 
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from  providers  and  receive  a  batch  number.    Initial  entry  of  all  claims  infor- 
mation onto  the  claims  file  is  completed,  then  claims  are  sent  to  NMPSRO  for 
numerical  diagnostic  and  procedure  coding.    If  a  claim  is  flagged  for  prepay- 
ment review,  it  is  given  to  a  NMPSRO  claims  examiner,  who  reviews  the  claim 
against  guidelines  and  checks  patient  history  through  a  computer  terminal.  If 
guidelines  are  violated  or  the  patient  history  looks  questionable,  the  claim 
is  referred  to  the  review  coordinator.    In  the  interim,  excepted  claims  are 
sent  to  Dikewood  for  second  entry  (complete  data  capture),  and  then  are  returned 
to  the  NMPSRO  review  coordinator  for  review  (designated  as  "pended"  claims). 

Precoded  claim  forms  are  separately  batched  for  entry  by  Dikewood.  The 
fiscal  agent  screener  examines  all  precoded  claims  prior  to  any  entry.  If 
problems  are  identified,  the  claim  receives  only  initial  entry.    If  no  prob- 
lems exist,  full  data  entry  is  attempted.    When  a  claim  cannot  be  completely 
entered,  the  data  entry  person  notes  the  problem  code  shown  on  the  computer 
terminal  and  routes  the  claim  for  correction.    If  claims  are  excepted  for 
medical  review  or  coding  purposes,  the  data  are  forwarded  to  NMPSRO.    If  such 
claims  are  not  excepted  (for  prepayment  screening,  eligibility,  problems,  etc.) 
full  data  entry  will  be  completed  and  microfilming  will  occur. 

Once  both  types  of  claims  are  fully  entered,  Dikewood  processes  them  for 
payment.    Beyond  the  programming  functions  involved  in  review  of  claims  for 
medical  appropriateness,  there  are  numerous  edits  and  screens  in  pricing  and 
payments  processing,  but  these  relate  only  to  Dikewood' s  functions  as  a  fiscal 
agent. 


ADDITIONAL  COMMENTS 

Although  NMPSRO's  ambulatory  review  system  is  fully  operational  (except 
for  the  fact  that  not  all  Title  XIX  physicians  are  using  precoded  forms),  NMPSRO 
is  still  "debugging"  the  system  and  making  minor  system  adjustments.  Further, 
several  NMPSRO  personnel  indicated  that  the  implementation  of  the  new  system 
was  beginning  to  strain  internal  physician  time  and  NMPSRO  executive  management 
of  the  program.    This  was  attributed  to  lack  of  knowledge  regarding  specific 
system  operations,  which  is  not  serious,  but  puts  NMPSRO  line  staff  personnel  in 
a  position  of  making  decisions  regarding  the  system  on  a  daily  basis.  Physicians 
and  top  NMPSRO  management  are  essentially  twice  removed  from  direct  system  opera- 
tions, and  HCMS  study  staff  could  not  document  routine  staff  meetings  between 
line  staff  and  top  management  in  order  to  assess  the  present  functioning  and 
future  direction  of  the  review  system. 
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SECTION  III 

HCMS     STUDY  INFORMATION 


NMPSRO  was  included  in  the  study  because  it  is  one  of  the  two  PSROs  conduc- 
ting systematic  review  of  Title  XIX  claims.    NMPSRO  agreed  to  participate  in 
the  study  in  discussions  with  HCMS  staff. 

The  following  documentation  has  been  received  from  NMPSRO: 

1.  NMPSRO  organizational  chart  (April  12,  1976) 

2.  Education  Committee  Action  Record  (form) 

3.  Ambulatory  Review  Cite  Sheet  (form) 

4.  Quality  Review  Program  (general  description  of  NMPSRO  ambulatory 
review  system  compiled  by  HCMS  staff,  April,  1976) 

5.  NMPSRO  Ambulatory  Review  Program  Budget  (August  1,  1975,  to  July  31,  1976) 

6.  NMPSRO  Ambulatory  Review  System  Description 

7.  Ambulatory  Review  Committee  minutes  (January  to  April,  1976) 

8.  Education  Committee  minutes  (September,  1975,  to  May,  1976) 

9.  Research  and  Development  Committee  minutes  (February,  1975,  to 
April,  1976) 

10.  Ad  Hoc  Coordinating  Committee  minutes  (January,  1976,  to  February, 
1976) 

11.  Flow  Diagram,  Dikewood  '307'  Form  Processing  (March  27,  1976) 

12.  HCMS  Evaluation  Plan:    Scope  of  Work  (revised,  August  31,  1975) 

13.  Health  and  Social  Service  Department  -  Title  XIX  claim  form 

14.  Status  Report,  Ambulatory  Care  Review  (February  3,  1976,  to  May  6, 
1976) 

NMPSRO  personnel  interviewed: 

1.  Executive  Director 

2.  Chairman,  Research  and  Development  Committee 

3.  Quality  Assurance  Program  Analyst 

4.  Review  Coordinator,  Ambulatory  Review  Program 

Related  personnel : 

5.  Acting  Director,  Social  Service  Agency,  State  of  New  Mexico 

6.  Senior  Program  Analyst,  Dikewood  Corporation 

7.  Research  Associate,  Health  Care  Management  Systems,  Inc. 

More  than  ten  hours  of  interview  time  were  spent  with  the  personnel  above, 
with  numerous  topics  covered: 

1.  Development  of  NMPSRO's  New  Ambulatory  Review  Program 

2.  NMPSRO  experience  in  EMCRO  affecting  development  of  new  review  system 

3.  Key  participants  in  the  development  of  present  system 

4.  Discussion  of  NMPSRO  review  system  in  great  detail 

5.  Key  personnel  who  operate  the  review  system 
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6.  Interaction  between  professional  committees  and  review  system  pro- 
cessing components 

7.  Interaction  between  professional  committees 

8.  Specific  tasks  of  NMPSRO  personnel  in  ambulatory  review 

9.  Role  of  Title  XIX  fiscal  agent  in  review  system 
10.    Management  problem  areas  in  review  system 
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BETH  ISRAEL  AMBULATORY  CARE  PROJECT 
Boston,  Massachusetts 


SECTION  I 
INTRODUCTION 


In  1969,  Beth  Israel  Hospital  joined  M.I.T.  Lincoln  Laboratories  to  im- 
plement the  Ambulatory  Care  Project.    The  Beth  Israel  Ambulatory  Care  Project 
(ACP)  was  contracted  to  develop  a  series  of  chronic  and  acute  medical  care 
protocols,  implement  those  protocols,  evaluate  their  effectiveness  and  train 
health  paraprofessionals  for  use  of  the  protocols.    The  project  was  conducted 
under  contract  between  the  Department  of  Health,  Education  and  Welfare  (DHEW) 
and  the  M.I.T.  Lincoln  Laboratory.    Beth  Israel  Hospital  has  been  a  full  sub- 
contracted collaborator  throughout  the  project.    The  project  was  conducted 
under  two  separate  contracts,  which  together  ran  from  June,  1969,  through 
February,  1976.    Funding  has  amounted  to  approximately  two  million  dollars 
over  seven  years. 

This  description  cannot  document  or  describe  each  protocol  that  was 
developed,  nor  the  number  and  types  of  studies  conducted  to  validate  the  pro- 
tocols.   The  objective  here  is  to  discuss  the  important  research-oriented 
components  of  the  ACP  in  developing  medical  protocols. 
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SECTION  II 


PROTOCOL  DEVELOPMENT 


CHRONIC  DISEASE  PROTOCOLS 

The  ACP  staff  consisted  of  electrical  and  computer  engineers  and  physi- 
cians, most  of  whom  had  joint  teaching  appointments  at  the  Harvard  Medical 
School  and  M.I.T.    These  skilled  technicians  and  clinically-oriented  specialists 
initially  felt  an  automated  computer  program  could  totally  administer  protocols. 
Prospective  protocol  logic  promised  to  be  complex,  and  they  felt  only  an  automated 
system  could  implement  them.    Most  of  the  time  between  1969  and  late  1970  was 
spent  in  system  design.    The  protocols  were  developed  by  physician  consultants, 
ACP  physicians  and  nurses.    Extensive  literature  reviews  and  a  multitude  of 
discussions  were  conducted  as  the  protocols  went  through  numerous  stages  of 
development.    ACP  personnel  indicated  that  during  the  same  period,  project 
personnel  began  to  have  reservations  regarding  total  automated  implementation 
of  all  protocols,  such  as: 

1.  Patient  needs;  the  chronically  ill  patients  especially  seemed  to 
need  human  interaction  in  addition  to  medication 

2.  The  need  to  perform  certain  components  of  a  physical  examination 

3.  Computer  systems  have  certain  "down"  periods  that  would  cause  severe 
problems  in  the  consistent  delivery  of  medical  care 

The  first  major  breakthrough  came  in  early  1971  when  ACP  staff  visualized 
a  way  in  which  logic  could  be  displayed  on  paper  for  manual  use,  then  subse- 
quently put  into  the  computer  to  analyze  whether  the  complicated  protocol  logic 
had  been  followed.    This  logic  was  pursued  and  in  1972  the  protocols,  previously 
developed  only  for  chronic  diseases,  were  transferred  from  the  total  computer 
system  to  the  manual  paper  system.    Two  studies  were  conducted  to  compare  the 
amount  of  time  spent  in  administering  the  computer  protocols  and  the  paper 
protocols.    Results  were  quite  similar. 

Throughout  the  automated  period,  the  financial  investment  was  substantial. 
With  the  need  for  less  computer  programming  and  the  future  direction  of  the 
protocols  decided,  the  financial  demands  were  lessened. 

Once  the  research  focus  had  been  more  definitely  determined,  ACP  staff 
concentrated  on  protocol  development.    The  contract  explicitly  identified  five 
chronic  medical  problems  for  protocol  development: 

1.  Diabetes 

2.  Hypertension 

3.  Cardiovascular 

4.  Prenatal 

5.  Pediatric 
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According  to  the  ACP  documentation,  a  protocol  is 

"an  instrument  which  describes  the  appropriate  steps  to  be  taken  in  manage- 
ment of  a  specific  disease  or  complaint  and  indicates  the  appropriate 
history,  physical  exam  and  lab  data  to  be  obtained.    Once  clinical  data 
are  obtained,  protocols  include  precise  rules  for  medical  action.  Proto- 
cols include  branching  logic  so  that  data  and  actions  are  individualized 
according  to  the  patient's  clinical  picture." 

ACP  staff  undertook  to  survey  specific  ambulatory  settings  to  determine  the 
frequency  of  visits  for  these  various  diagnostic  categories.    The  surveys  indi- 
cated "most  common"  chronic  conditions  were  hypertension,  diabetes,  arterio- 
sclerotic heart  disease,  obstructive  lung  disease,  arthritis  conditions  and 
gastrointestinal  disturbances.    ACP  staff  confirmed  the  surveys  with  several 
independent  studies. 

After  determining  key  variables  in  treatment  of  chronic  disease,  ACP  staff 
devised  protocols  for  hypertension,  cardiovascular  disease  and  diabetes,  assuming 
a  positive  diagnosis  had  been  made.    Each  protocol  went  through  an  extensive 
validation  process  in  four  areas:* 

1.  Medical  safety 

2.  Operational  acceptability 

3.  Physician  acceptance 

4.  Patient  acceptance 

Each  validity  area  had  several  research  components  and  studies  for  deter- 
mining validity.    Documentation  notes  each  protocol  had  to  contain  or  do  the 
fol lowing: 

1.  List  of  data  (symptoms,  signs,  test  results,  etc.)  to  be  collected. 
Occasionally,  the  explicit  phrasing  of  questions  posed  to  the  patient 
must  be  included. 

2.  Indicate  the  circumstances  under  which  each  datum  should  be  collected, 
(e.g., certain  tests  should  be  performed  only  for  certain  symptomatic 
indications) . 

3.  Allow  space  for  date  to  be  recorded  as  collected.    These  data  must  be 
displayed  in  some  fashion  that  facilitate  review  by  physician. 

4.  List  actions  to  be  taken  by  protocol  user  (therapies,  referrals,  in- 
structions to  the  patient,  etc.). 

5.  Show  the  protocol  user  how  to  infer  from  the  data  which  of  the  above 
actions  are  appropriate. 

6.  Simplify  format  as  much  as  possible. 

At  the  conclusion  of  this  process,  documentation  notes  that  a  full-scale  proto- 
col system  began  in  1973  at  Boston  City  Hospital  Diabetes  Clinic  and  Beth  Israel 
Medical  Clinic.    During  the  implementation,  all  patients  presenting  the  identi- 
fied chronic  conditions  were  first  evaluated  by  health  assistants. 

Concurrently  with  the  operational  activities,  ACP  personnel  developed  an 
evaluation  plan  to  measure  cost  and  time  resource  changes  for  the  setting  which 
implemented  the  protocol  management  system.    The  evaluation  compared  an  experi- 
mental period  when  protocols  were  implemented  with  other  periods  when  no  protocols 


*For  results  of  studies  refer  to  Komaroff,  A.L.,  et  al.,  NEjM  290:307-312  1974 
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were  operating.    The  evaluation  examined  process  and  outcome  measures  of 
quality,  and  efficiency  questions  dealing  with  amount  of  physician  time  saved 
by  employing  health  assistants  using  the  protocols. 

In  1975,  the  implementation  of  protocols  moved  away  from  the  health 
assistants  to  the  nurse-practitioners  and  more  formally  trained  physician 
assistants.    ACP  personnel  indicated  the  change  was  made  for  two  basic  reasons: 
(1)  nurses  who  had  formalized  nursing  education  were  being  somewhat  supplanted 
by  health  workers  without  education;  (2)  nurses  seemed  to  want  to  implement 
protocols  because  they  were  underutilized  and  it  presented  a  challenge  to 
their  training.    At  present,  however,  health  assistants  in  other  clinics  con- 
tinue to  use  protocols. 


ACUTE  ILLNESS  PROTOCOLS 

Development  of  the  acute  illness  protocols  began  in  1972.    The  chronic 
disease    protocols  had  been  designed  to  guide  the  care  of  patients  whose  ill- 
nesses had  already  been  diagnosed  by  physicians.    In  contrast,  patients  seeking 
care  for  new  acute  illness  symptoms  might  never  have  been  so  diagnosed.  This 
raised  two  fears: 

1.  If  a  nonphysician  uses  the  protocol,  he  or  she  may  overlook  important 
information  a  physician  would  be  aware  of,  that  the  protocol  may 
have  missed. 

2.  Patients  do  not  always  reveal  the  real  reason  they  want  medical 
attention,  and  a  nonphysician  may  be  insensitive  to  this. 

The  new  contract  beginning  in  1972  required  ACP  to  develop,  validate  and 
document  protocols  for  the  following: 

1.  Triage 

2.  Urinary  tract  infection  and  vaginitis 

3.  Headache 

4.  Chest  pain 

5.  Acute  gastrointestinal  distress 

6.  Upper  respiratory  infection 

7.  General  gynecology 

8.  Rash 

9.  Low  back  pain 

Again,  the  ACP  had  surveyed  several  ambulatory  settings  to  determine  the 
frequency  of  certain  ambulatory  complaints;  the  results  of  this  survey  led  to 
the  identification  of  those  symptoms  specified  by  the  contract.  Protocols 
were  then  developed  for  all  problems  outlined  in  the  contract  by  several  ACP 
study  teams  headed  by  one  general  internist  familiar  with  primary  care.  The 
same  process  that  accompanied  the  total  development  of  chronic  protocols  was 
duplicated  for  the  acute  problems. 

Protocol  development  is  an  essential  feature  of  the  ACP  research  and  imple- 
mentation efforts.    ACP  personnel  provided  investigators  with  a  complete  outline 
used  in  protocol  development.    This  extensive  process  is  outlined  below  and  notes 
series  of  stages  in  development  and  implementation  (for  testing)  of  the  protocols. 
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Protocol  Development 

A.  Outline  of  the  intentions  of  protocol: 

1.  Specify  what  presenting  complaints  should  be  addressed  by  the 
protocol  (entry  conditions),  and  modify  accordingly 

2.  Specify  what  conditions  the  protocol  "cannot  afford  to  miss" 

and  define  the  history,  physical  and  lab  data  most  likely  to  pick 
them  up 

3.  Specify  the  conditions  the  protocol  "can  afford  to  miss" 

4.  Specify  the  circumstances  which  warrant  referral  to  the  physician 

5.  Specify  the  conditions  the  protocol  is  expected  to  diagnose  and 
treat 

6.  Define  the  contraindications  to  therapy  and  state  alternative 

B.  Identify  prior  or  analogous  protocols  developed  by  others  and  compare 

C.  Lay  out  first  version  of  the  protocol,  specifying  clinical  data  to  be 
collected  and  medical  judgments  to  be  made,  if  deviations  from  work 
of  others  is  significant 

D.  Prepare  standing  orders  for  therapy  and  instructions  for  follow-up 

E.  Obtain  peer  review  of  above  by  both  "experts"  and  practitioners,  with 
possible  subsequent  revisions 

F.  Perform  initial  trials  to  highlight  major  problems  with  the  protocol 

G.  Revise  protocol  to  a  "stable"  form  which  will  remain  unchanged  during 
formal  validation  study 

H.  Assure  that  the  typed  copy  of  the  "stable"  protocol  is  accurately 
laid  out 


Protocol  Validation 

A.  Find  the  "right"  sites  (e.g.  relatively  high  volume,  manageable) 

1.  Assess  volume  of  patients  with  protocol  illness 

2.  Assess  availability  of  paramedics  to  use  protocols  -  either  already 
on  site  or  through  ACP 

3.  Assess  adequacy  of  the  necessary  laboratory  backup 

4.  Assess  availability  of  and  recruit  backup  physican(s)  both  to  con- 
firm data  collected  by  paramedic  and  to  assess  wisdom  of  medical 
judgment  made  by  protocol. 

5.  Establish  training  time  available  for  protocol  users 

B.  Get  validation  under  way 

1.    Explain  purpose  of  validation  to  all  involved  physicians,  nurses, 
paramedics,  and  administrators 
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2.  Arrange  for  training  of  nonphysician  protocol  user(s) 

3.  Establish  mechanism  for  having  backup  physicians  assess  accuracy 
of  clinical  data  collected  by  nonphysicians 

4.  Establish  mechanism  for  checking  whether  protocol  logic  has  been 
followed  accurately  (computer  can  do  this) 

5.  Establish  mechanism  for  having  backup  physicians  comment  on  proto- 
col therapy  and  disposition  decisions,  or  indicate  clearly  their 
own  decisions  (for  comparison  with  protocol) 

6.  Establish  mechanism  for  comparing  protocol  judgments  with  objec- 
tive measure  of  their  appropriateness,  e.g.,  lab  results 

7.  Establish  mechanism  for  assessing  positive  and  negative  clinical 
outcome  by  delayed  chart  pulls,  telephoning  patients,  etc.;  specify 
criteria  for  measuring  clinical  outcome  (symptomatic  relief,  objec- 
tive remission,  need  for  alternative  care) 

8.  Assess  which  protocol  attributes  or  questions  are  particularly 
valuable,  and  which  appear  extraneous 

9.  Assess  protocol  rate  of  referral  to  physician,  and  extent  of  both 
under-  and  overreferral s 

10.  If  during  formal  validation  study  the  need  becomes  apparent  for 
major  changes  in  protocol  content  or  decision  logic,  give  serious 
consideration  to  starting  validation  study  over  again 

11.  Prepare  report  of  validation  study  outcome 

(a)  for  internal  publication,  or 

(b)  possibly  for  journal  publication 


Protocol  Package  Development 
Preparation  of: 

1.  Introductory  section,  directed  to  physician 

2.  Explanation  of  how  to  follow  the  protocol's  logic  scheme 

3.  Medical  rationale,  directed  to  physician,  explaining,  attribute-by- 
attribute,  question-by-question,  why  protocol  contains  or  excludes 
certain  clinical  data,  and  why  it  makes  the  diagnostic,  therapy,  and 
disposition  judgments  it  does 

4.  Patient  education  handout 

5.  Training  manual ,  designed  to  instruct  the  would-be  user  of  the  protocol, 
containing  descriptions  of  the  relevant  anatomy,  physiology,  and  patho- 
physiology, explanation  of  the  protocol  content  (simpler  than  medical 
rationale)  including  how  to  ask  certain  questions,  explanation  of  how 

to  perform  physical  examination  and  laboratory  tests  called  for  by  the 
protocol,  and  glossary  of  terms 


ACP  personnel  indicated  major  resources  were  spent  in  developing  the  proper 
format  for  the  protocols.    As  indicated  earlier,  a  major  decision  by  ACP  per- 
sonnel was  to  place  the  protocols  in  a  manually  written  format.    Once  this  was 
accomplished,  staff  had  to  concentrate  on  the  exact  format  for  each  protocol. 

ACP,  after  much  study  and  experimentation,  arrived  at  a  final  protocol 
format  which  allows  "departures  from  a  simple  sequential  order  of  events  and 
allows  for  complex  branching." 
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Implementation  of  protocols  in  a  variety  of  settings  was  an  important 
consideration  to  the  ACP  researchers.    Successful  implementation  would  require 
that  protocols  be  functional  to  health  providers  and  administrators  in  provi- 
sion of  medical  services.    Because  use  of  protocols  develops  an  extensive  data 
base,  this  data  base  can  assess  aggregate  performance  of  providers  utilizing 
protocols.    An  example  of  this  type  of  activity  is  the  production  of  summary 
statistics  and  interrogation  of  the  data  base.    An  example  of  this  activity 
is  the  following: 


1.  Summary  Statistics:    Aggregate  data  are  provided  monthly  for  each 
protocol  by  type  of  provider  and  type  of  protocol  user. 

2.  Summary  Slot  Statistics:    Aggregate  data  are  provided  monthly  for 
each  protocol  by  the  number  of  errors  for  every  question  of  the 
protocol . 

3.  Interrogation  of  the  Data  Base:    Using  the  data  retrieval  program, 
it  is  possible  to  search  the  data  base  for  the  following  questions: 

a.  How  frequently  do  various  complaints  occur  for  each  protocol? 

b.  How  frequently  do  various  symptoms  occur,  either  singly  or  in 
conjunction  with  others? 

c.  How  often  do  various  physical  exams  or  lab  test  findings  occur? 

d.  How  do  certain  findings  or  clusters  of  findings  relate  to  a 
given  defined  diagnosis  end-point? 

This  program  also  answers  these  developmental  questions: 

a.  For  a  given  medical  problem  in  a  homogeneous  population,  when 
findings  of  users  vary  significantly,  does  this  suggest  inappro- 
priate use  of  protocol  by  some  users? 

b.  What  branch  points  in  the  protocol  constitute  major  reasons  for 
referring  patients  to  physicians,  and  can  this  referral  rate  be 
reduced? 


Combining  the  data  base  information  with  other  observations,  it  is  possible  to 
determine  how  medical  treatment  resources  were  expended  by  answering  the  following: 

1.  How  much  direct  professional  time  is  used  to  care  for  a  patient? 

2.  What  proportion  of  total  costs  are  expended  on  medicine,  protocol - 
initiated  lab  tests,  and  professional  time? 

Thus  far,  interrogation  data  base  analysis  have  been  tested  for  the  upper 
respiratory  infection  and  urinary  tract  infection-vaginitis  protocols. 


Audi  table  Checkl ist 

As  indicated,  ACP's  critical  concern  is  the  successful  implementation  and 
use  of  the  protocols.    While  the  protocols  facilitate  auditing  of  medical  care 
because  the  structured  recording  of  clinical  findings  guide  practitioners  in 
care,  ACP  personnel  discovered  that  a  key  to  successful  use  of  protocols  was 
elimination  of  certain  disadvantages  in  the  initial  training  and  use  of  protocols. 
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ACP  indicates  that  one  method  for  collecting  the  same  type  and  quality  of 
information  using  the  protcol  is  to  introduce  the  use  of  an  audi  table  check- 
list, after  practitioners  have  become  familiar  with  the  protocols.    Like  a 
protocol,  it  is  a  checklist,  but  unlike  the  protocol,  the  logic  is  not  dis- 
played. 

The  audi  table  checklist  is  an  encounter  form  designed  for  a  particular 
medical  problem,  as  opposed  to  forms  for  which  information  on  any  condition 
is  recorded.    Information  on  an  audi  table  checklist  is  recorded  in  a  format. 
It  does  not  involve  duplication  of  recording  and  serves  as  a  progress  report 
for  an  encounter.    It  is  filed  in  the  medical  record  and  a  duplicate  copy  can 
be  used  for  audit  purposes.    Space  is  available  on  the  checklist  for  additional 
information.    Subjective  data  and  objective  data  are  grouped  separately,  and 
there  is  a  section  to  indicate  ordered  laboratory  tests,  diagnostic  impres- 
sion(s)  and  the  plan(s).    For  most  data,  a  "yes"  or  "no"  is  required.  The 
computer  performs  an  audit  using  the  checklist  to  identify  various  types  of 
errors  as  does  the  audit  protocol.    However,  this  checklist  audit  allows  a 
more  precise  description  of  the  type  of  error.    The  types  of  error  listed  are 
as  follows: 


1 .  Missing  data 

2.  Inappropriate  lab  test 

3.  Missing  lab  test 

4.  Inconsistent  impression 


5.  Missing  impression 

6.  Inconsistent  plan 

7.  Missing  plan 

8.  Missing  physician  consult 


PROJECT  RESULTS  AND  CONCLUSIONS 

A  series  of  prospective,  randomized  controlled  studies  have  been  performed 
to  assess  the  quality,  efficiency  and  cost  of  care  in  a  system  where  practitioners 
other  than  physicians  use  protocols.    These  have  been  published  in  several  major 
medicine  and  nursing  journals.    The  studies  have  been  conducted  in  seven  dif- 
ferent clinics  over  a  three-year  period.    The  general  conclusions  are  as  follows: 

1.  Quality  of  care  as  reflected  by  process  criteria  is  at  least  as  good 
in  the  protocol  system 

2.  Outcome  of  care  (symptom  relief,  objective  evidence  of  cure)  is  at 
least  as  good 

3.  Patients  with  serious  illness  have  not  been  "overlooked,"  but  have  been 
appropriately  referred  to  physicians 

4.  Patient  satisfaction  is  at  least  as  good 

5.  Physician  time  with  patients  has  been  reduced  50  to  85%  in  acute  ill- 
nesses, 25  to  40%  in  chronic  illnesses 

6.  New  practititioner  time  averages  20%  more  per  patient  encounter  than  did 
physician  time 

7.  System  cost  (manpower,  test  ordering,  medication  ordering)  is  the 
same  or  less  than  the  traditional  physician  system 

In  terms  of  distributing  protocol  material,  as  of  February,  1976,  15,022 
copies  of  the  ACP  protocols  and  1,630  copies  of  support  materials  have  been 
sold  to  more  than  300  private  practices,  hospital  outpatient  departments,  com- 
munity clinics,  HMOs,  and  training  programs  in  40  states  and  12  foreign 
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countries  (a  book  entitled:    Common  Acute  Illnesses:    A  Problem-Oriented  Text- 
book with  Protocols  will  be  published  in  Spring,  1977,  by  Little,  Brown  and 
CoTT 

The  training  activities  are  viewed  as  essential  by  ACP  personnel.  Proper 
use  of  protocols,  according  to  ACP  personnel,  determines  the  success  or  failure 
of  the  implementation  within  a  particular  setting. 

Training  programs  in  protocol  usage  presently  exist  at  the  Beth  Israel  Hos- 
pital Walk-in  and  the  Dimock  Street  Health  Center.    Although  the  program  is 
informal  in  nature,  ACP  staff  feel  the  clinical  benefits  of  the  protocol  metho- 
dology are  such  that  they  strongly  determine  the  impact  of  implementation. 
According  to  ACP  personnel,  the  training  is  not  so  much  a  function  of  education, 
but  of  building  confidence  in  the  ability  to  provide  medical  care.    Each  trainee 
is  provided  with  a  set  of  supporting  materials  including  an  introduction,  a 
training  manual  and  patient  education  materials.    The  introduction  includes 
a  description  of  the  use  and  rationale  of  the  logic  for  each  protocol  in  addition 
to  numerous  references  to  the  literature.    The  training  manual  describes  in  basic 
language  the  anatomy  and  physiology  involved  as  well  as  the  procedures  for  taking 
a  patient's  history,  performing  a  physical  exam  and  conducting  lab  tests.  The 
patient  education  materials,  written  in  lay  language,  discuss  the  nature  of  the 
condition,  the  purpose  of  the  treatment,  the  potential  side  effects,  the  need 
for  compliance  with  treatment  and  follow-up.    As  noted  earlier,  the  work  at 
these  two  settings  is  being  monitored  and  systematically  audited  by  ACP  personnel. 


230-045  O  -  77  -  21 
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SECTION  III 


HCMS     STUDY  INFORMATION 


BIACP  agreed  to  participate  in  the  survey  following  HCMS  correspondence 

in  October,  1975.    Study  staff  conducted  a  site  visit  on  May  3,  1976.  The 

documentation  received  and  compiled  before  and  after  the  site  visit  includes 
the  following: 


1.  Komaroff,  Anthony  L.,  et  al . :    Quality  efficiency,  and  cost  of  a 
physician-assistant-protocol  system  for  management  of  diabetes 
and  hypertension.    Diabetes,  25:297-306  April  1976. 

2.  Komaroff,  Anthony  L.,  et  al . :    Nurse  practitioner;  management  of 
common  respiratory  and  geniturinary  infections  using  protocols. 
Nursing  Research,  25:84-89  March-April  1976. 

3.  List  of  questions  of  importance  in  setting  up  protocols  asked  of 
recipients  of  protocols. 

4.  Stages  in  Creating  A  Protocol. 

5.  The  Beth  Israel  Hospital  Ambulatory  Care  Project  1969-1976,  Final 
Contract  Report  Executive  Summary  (15-A)  (February  26,  1976) . 

6.  Beth  Israel  Hospital  Progress  Report  Ambulatory  Care  Project  -  II 
(13A)  (August  1,  1975). 

7.  Beth  Israel  Hospital  Final  Contract  Report  1969-1976  (14A) 
(February  29,  1976). 

8.  Various  computerized  printouts  of  statistical  analyses  of  protocol 
implementation  input  (September  and  October  1975). 

9.  List  of  most  frequent  diagnostic  codes  encountered  by  staff  during 
protocol  implementation  with  their  respective  HICDA  codes. 

10.    Introduction,  Training  Manual  and  Patient  Education  Materials  of 
upper  respiratory  infection  and  urinary  tract  infection  protocols. 


During  the  site  visit,  study  staff  interviewed  three  persons  connected 
with  BIACP: 


1.  Medical  Director  of  the  Ambulatory  Care  Project 

2.  Physicist  and  Systems  Analyst  responsible  for  all  activity  of  the 
ACP  having  to  do  with  computer  quantitative  analysis 

3.  Nurse  Practitioner  at  the  Dimock  Street  Health  Center 


Topics  covered  during  the  interview  were  as  follows: 


1.  Historical  development  and  implementation  of  protocols 

2.  Constraints  in  the  development  with  regard  to  human  fears  and 
political  issues 

3.  Funding  sources 

4.  Training  approach  in  protocol  use 

5.  Staff  support  and  responsibilities 

6.  The  transition  from  computerized  protocols  to  manual  paper  protocols 

7.  Audits  or  protocols  (routine  and  special  studies) 

8.  Misconceptions  about  protocols 

9.  Output  reports  for  comparative  provider  assessment  and  provider 
sel f-assessment 
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10.  Constraints  in  getting  nonprotocol  users  to  use  protocols 

11.  Questionnaires  administered  to  protocol  users  outside  Beth  Israel 
Hospital 

12.  Current  protocol  implementation  and  validation  studies 
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MULTNOMAH  FOUNDATION  FOR  MEDICAL  CARE 
Portland,  Oregon 


SECTION  I 
INTRODUCTION 


The  Multnomah  Foundation  for  Medical  Care  (MFMC)  was  initially  organized 
in  1961  by  a  group  of  solo  practitioners  and  small  group  practitioners  to  pro- 
vide insured  individuals  with  indemnity  benefits  competitive  with  those  being 
offered  by  other  groups  (e.g.,  closed  panel  groups).    The  major  corresponding 
purpose  was  to  establish  a  systematic  fee  schedule  on  a  community-wide  basis. 
Since  1970,  however,  the  MFMC  has  changed  its  thrust  to  providing  review  of 
medical  care  for  the  community  in  both  operational  and  developmental  programs. 

MFMC  received  Experimental  Medical  Care  Review  Organization  (EMCRO)  funding 
between  June,  1971  and  May,  1974,  and  was  included  in  the  study  because  of 
significant  and  unique  experience  developing  and  implementing  certain  types 
of  ambulatory  quality  assurance  activities. 

MFMC  was  established  in  1963  by  the  Multnomah  County  Medical  Society. 
The  foundation  was  soon  disbanded,  however,  due  to  physician  inactivity  and  lack 
of  interest.    In  1965,  the  medical  society  created  the  Health  Insurance  Review 
Committee  to  review  cases  for  Part  B  Medicare  carriers  and  opened  a  complaint 
department  to  all  carriers  who  questioned  claims  received  from  members  of  the 
society.    Throughout  this  period,  the  committee  experienced  multiple  frustra- 
tions including:    (1)  the  inefficiency  of  committee  process  for  reviewing 
medical  fee  questions,  (2)  the  inability  to  confront  questions  related  to 
appropriateness  of  care,  and  (3)  a  growing  concern  that  the  committee  was  merely 
serving  as  an  extension  of  the  insurance  carriers'  fiscal  operations. 

As  a  result  of  these  problems,  two  representatives  of  the  society  attended 
an  annual  meeting  of  the  American  Association  of  Foundations  for  Medical  Care 
in  1969;  they  returned  to  Portland  with  an  enthusiastic  report  to  the  Multnomah 
County  physicians  about  the  progress  of  California  foundations,  and  the  issue 
of  reactivating  the  Multnomah  Foundation  was  again  brought  up  before  society 
leadership.    Documentation  notes  the  argument  for  reactivating  the  Foundation 
centered  around  moving  the  medical  community  in  Portland  from  a  position  of 
simply  hearing  complaints  to  becoming  a  strong  positive  force  for  improvement 
of  medical  care  by  performing  quality  review. 

During  1970,  the  medical  society's  limited  executive  staff  allowed  little 
room  for  Foundation  development;  however,  there  were  a  few  "brainstorming" 
sessions  among  the  most  interested  physicians.    Then  a  letter  from  the  National 
Center  for  Health  Services  Research  (headed  by  Dr.  Paul  Sanazaro  at  the  time) 
was  received  requesting  proposals  for  Experimental  Medical  Care  Review  Organiza- 
tions (EMCRO).    The  more  interested  physicians  saw  an  opportunity  to  begin 
developing  a  serious  review  system  with  EMCRO  funding. 
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MFMC  responded  to  the  request  with  a  proposal  to  design  and  operate  a 
system  for  assessing  the  appropriateness  and  necessity  of  certain  care.  In 
June,  1971,  EMCRO  funds  were  awarded  primarily  for  development  of  methodologies 
to  assure  patients' receipt  of  good  medical  services  from  private  practitioners. 

There  were  seven  main  objectives  in  the  EMCRO  proposal: 

1.  To  develop  criteria  for  different  diagnoses  and  treatment  procedures 

2.  To  develop  a  sample  base  of  patients 

3.  To  develop  a  minimum  sample  group  (250  as  required)  of  providers 

4.  To  develop  tools  to  collect  accurate  and  timely  medical  care  treat- 
ment information 

5.  To  develop  an  automated  system  to  accept  patient  care  data 

6.  To  develop  techniques  and  organizational  capabilities  for  assessment 
of  actual  care  rendered 

7.  To  develop  methods,  techniques,  and  organizational  capabilities  for 
achieving  behavioral  practice  changes  in  physicians,  according  to 
results  of  assessments 

After  three  years  of  operational  EMCRO  experience,  MFMC  personnel  began 
developing  a  new  health  service  data  support  system  entitled  the  "Multi-Use 
Medical  Care  Data  System."    Many  lessons  and  experiences  based  on  EMCRO  programs 
were  incorporated  in  the  development  of  the  multi-use  system,  including  a  broader 
use  for  one  medical  data  base.    MFMC's  EMCRO  experience  led  to  development  of 
inpatient  utilization  review  components,  the  Concurrent  On-Site  Evaluation  and 
Review  Effort  (CONSERVE)  program.    The  documentation  indicates  this  program  was 
put  in  operation  and  sold  to  six  insurance  carriers  and  eight  hospitals.  These 
developments  led  to  MFMC's  designation  in  1974  as  a  conditional  Professional 
Standards  Review  Organization  (PSRO). 

In  1974,  MFMC  applied  to  the  Bureau  of  Health  Services  Research  (BHSR) 
for  grant  funding  to  research,  develop  and  implement  the  multi-use  medical 
data  system,  and  these  operations  are  the  focus  of  this  description.    The  pro- 
posal to  Health  Services  Research  described  the  design,  development  and  testing 
of  the  multi-use  data  system  to  meet  specific  needs  of  various  users  of  health 
information.    Specific  objectives  were: 

1.  To  decrease  the  load  of  paperwork  and  other  indirect  patient  care 
tasks  performed  by  physicians 

2.  To  provide  increased  uniformity  of  data  recording 

3.  To  implement  a  means  of  data  capture  at  the  source  of  care 

4.  To  provide  improved  quality  of  care  through  a  series  of  subproposals 
based  on  results  of  quality  assurance  activities 

MFMC's  strong  point  in  applying  for  funding  of  the  multi-use  project  was 
its  experience  in  implementing  the  EMCRO  study.    One  MFMC  document  developed 
this  logic  in  a  section  called  "Lessons  from  EMCRO."    MFMC  staff  documented  11 
factors  learned  through  experience  which  would  dictate  certain  approaches  to 
consider  in  developing  the  multi-use  data  system: 

1.  Need  for  total  feedback  to  physician 

2.  Redesign  of  the  encounter  form 
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3.  A  new  encounter  form  forces  the  physician  to  slow  down  and  write 
more  legibly 

4.  The  patient  profile  is  the  most  often  requested  computer  output  at 
EMCRO 

5.  As  EMCRO  output  increased,  turn-around  time  decreased 

6.  As  on-line  system  is  desirable;  the  additional  costs  of  such  a  system 
on  an  operational  basis  are  balanced  by  the  efficiency  of  faster  turn- 
around time 

7.  The  on-line  system  has  a  much  shorter  elapsed  time  for  data  capture 

8.  The  review  system  does  not  have  evaluation  mechanisms  built  in;  hence 
objective  evaluation  was  difficult 

9.  The  on-line  system  has  few  steps 

10.  The  problem  codes  need  refinement 

11.  Average  rate  of  use  of  the  entire  medical  care  data  portion  of  the 
encounter  form  was  20% 

Once  MFMC  received  the  BHSR  grant,  it  was  implemented  immediately.  There 
were  some  problems  with  project  implementation;  this  description  will  focus  not 
on  operational  status  but  on  the  research  and  developmental  elements  of  the 
program. 
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SECTION  II 

QUALITY     ASSURANCE  PROGRAM 


The  experience  of  the  EMCRO  program  provided  MFMC  direction  in  developing 
a  new  area-wide  approach  to  data  collection  for  quality  assurance  (or  other 
health-related  programs).    MFMC's  subjective  evaluation  of  EMCRO  revealed  systems 
such  as  EMCRO  placed  a  significant  burden  on  physicians  to  complete  necessary 
data  collection  procedures.    The  multi-use  system,  therefore,  is  designed  to 
simplify  these  procedures  by  devising  a  set  of  all  inclusive  encounter  forms 
for  both  group  and  solo  physicians. 

The  initial  phase  of  program  development  required  MFMC  personnel  to  deter- 
mine issues  which  should  be  addressed  by  the  multi-use  system.    Four  areas  of 
need  were  identified: 

1.  A  task  force  of  physicians  to  determine  MFMC  needs 

2.  Examination  of  other  data  systems 

3.  Literature  research 

4.  Needs  already  identified  through  EMCRO  experience 

After  the  initial  list  was  generated,  MFMC  personnel  and  other  health  pro- 
fessionals (physician,  health  insurance  representatives,  etc.)  distilled  a  list 
of  priorities  to  be  addressed  by  the  multi-use  system: 

1.  Development  of  efficient  multi-purpose  data  capture  method 

2.  Development  of  professional  review  (patient  and  provider  profiles) 
procedures 

3.  Collection  of  quality  assurance  statistics 

4.  Collection  of  medical  statistics 

5.  Collection  of  health  policy  foundation  and  statistics 

6.  Establishment  of  a  computer-based  medical  management  information 
system 

7.  Scheduling  health  maintenance  data  collection 

8.  Establishment  of  insurance  claims  generator 

9.  Establishment  of  a  billing  system 

10.    Design  of  a  patient  medical  history  summary 

MFMC's  major  emphasis  was  development  of  data  collection  instruments  for 
selected  physicians  (both  group  and  solo)  to  test  the  most  critical  aspect 
of  the  multi-use  system,  the  data  collection  procedures.    Documentation  notes 
the  data  collection  system  would  have  to  "simplify  participating  providers' 
overall  paperwork  while  providing  them  with  practice  profiles  for  management" 
and  other  uses. 

MFMC  personnel,  along  with  the  multi-use  system  contractor,  the  Dikewood 
Corporation,  developed  four  basic  data  collection  forms: 

1.  Practice  and  physicians  enrollment  form 

2.  Patient  data  form 
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3.  Patient  medical  history 

4.  Pediatrics  physician-patient  encounter  form 

Forms  No.  1  and  No.  2  are  essentially  multi-use  data  forms  developed  to 
allow  the  system  to  collect  in  a  computer  file  at  Dikewood  both  provider  and 
patient  information  as  an  integral  part  of  multi-use.    Each  enrollment  form 
required  each  provider  and  patient  to  have  discrete  multi-use  numbers.  The 
provider  form  could  incorporate  data  from  a  single  physician  or  a  group  prac- 
tice of  three  physicians.    The  provider  data  required  included: 

1.  Specialty  or  subspecialty 

2.  License  number 

3.  Name 

4.  Signature 

5.  Date  of  birth 

Practice  information  included: 

1 .  Name 

2.  Address 

3.  Census  tract  number 

4.  Type  of  practice 

5.  Office  representative 

The  form  required  MFMC  coder  identification  and  batch  control  number. 
The  patient  data  form  required  the  following: 

1 .  Patient  name 

2.  Address 

3.  Census  tract  number 

4.  Payment  source  information 

5.  Primary  and  secondary  insurance  policy  data 

This  final  section  of  the  form  required  several  additional  pieces  of  informa- 
tion including  insured's  identification  number,  insurance  company,  name  and 
address  and  insured's  name  and  address. 

Two  other  encounter  forms,  No's.  3  and  4,  were  designed  to  capture 
physician-patient  encounter  information  to  complement  enrollment  data.  Ac- 
cording to  MFMC  personnel,  these  two  forms  required  a  great  amount  of  initial 
development  time  and  cost,  but  once  the  general  form  and  content  were  defined, 
the  forms  were  readily  modifiable  to  other  practices.    Each  form  has  numerous 
descriptive  medical  data  elements,  and  corresponding  boxes  to  check  when  such 
descriptive  data  have  to  be  recorded.    Each  form  also  has  a  carbon  copy,  where 
data  are  simultaneously  recorded  when  a  descriptive  data  element  is  checked  on 
the  original.    The  copy  form  has  corresponding  numerical  data  codes  for  each 
descriptive  data  element,  requiring  a  provider  simply  to  check  the  descriptive 
elements  once. 

To  illustrate  the  format  for  data  collection,  assume  the  Pediatrics  form 
were  to  be  used  for  a  specific  encounter.    General  patient  information  would 
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be  recorded  (patient  name,  multi-use  number,  date  of  birth  and  encounter  date), 
and  if  necessary,  problems  or  diagnosis,  laboratory  work,  diseases  or  problems 
treated  and  immunizations.    If  a  throat  culture  had  been  ordered,  the  box  next 
to  that  descriptive  element  would  have  two  pieces  of  information  entered:  (1) 
a  number  indicating  what  problem  the  culture  related  to  (noted  in  previous  section 
of  form)  and  (2)  a  checkmark  indicating  the  procedure  was  ordered.    When  this 
information  is  marked,  the  numerically  coded  sheet  below  the  original  is  simul- 
taneously marked.    Both  encounter  forms  have  areas  where  a  provider  can  enter 
free  text  information.    The  coding  of  this  information,  as  well  as  coding  checks, 
is  completed  by  MFMC  data  clerks. 

The  Pediatric  form  was  developed  in  conjunction  with  several  pediatricians 
and  one  nurse  in  the  area.    Each  form  went  through  numerous  revisions  and  was 
implemented  in  three  pediatric  offices. 

While  the  data  collection  forms  were  being  designed,  MFMC  began  developing 
the  automated  processing  system  necessary  for  the  multi-use  system.    The  Dike- 
wood  Corporation  was  engaged  to  contribute  to  system  development  and  processing 
for  the  multi-use  system  (Dikewood  had  been  the  systems  contractor  for  the  EMCRO 
project) . 

Dikewood 's  main  activities  centered  on  designing  data  input  formats  and  a 
series  of  reports.    The  developed  (operational)  reports  displayed  to  study  staff 
included  Diagnosis  and  Profile  and  Diagnoses  Summary  Reports.    Each  report  was 
physician  specific,  covering  a  specified  period  of  time.    The  Diagnosis  and 
Profile  report  included  the  following  classes  of  information: 

1.  Patient  encounter  data 

2.  Patient  age  data 

3.  Aggregated  services 

4.  Detailed  services  data,  including  laboratory  and  X-ray 

5.  Medication  data 

6.  Other  therapeutic  procedure  data 

7.  Description  of  disposition 

There  were  many  subcategories  of  data  with  corresponding  number  and  percentages, 
as  well  as  distinctions  between  male  and  female. 

The  Diagnoses  Summary  Report  (DSR),  also  physician  specific,  covers  a  speci- 
fied period  of  time.    This  report  presents  aggregated  figures  for  the  same  several 
categories  in  the  profile  report: 

1.  Patient  encounter  data 

2.  Patient  age  data 

3.  Diagnosed  patient  data 

4.  Undiagnosed  patient  data 

5.  Aggregated  service  data 

6.  Detailed  service  data 

The  next  section  of  the  DSR  includes  the  diagnoses  (by  H-ICDA  codes)  by 
frequency,  with  a  corresponding  abbreviated  description,  patient  percentages 
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and  encounter  data.  These  two  reports  were  the  onTy  ones  developed  at  the  time 
of  on-site  investigations  by  study  staff. 

The  next  task  faced  by  MFMC  personnel  was  to  recruit  providers  to  partici- 
pate in  the  program  employing  the  data  collection  format  in  their  practices  and 
allowing  MFMC  to  aggregate  data  and  use  the  resulting  reports.    The  original 
research  design  required  16  provider  offices  to  be  included  in  the  implementa- 
tion; however,  MFMC  experienced  some  problem  in  recruiting  providers,  and  at 
last  report,  only  three  physicians  were  participating  in  multi-use  implementa- 
tion. 


ADDITIONAL  COMMENTS 

MFMC  personnel  noted  implementation  of  the  multi-use  system  is  proceeding 
slower  than  anticipated.    There  were  two  reasons  cited: 

1.  Physicians  were  unwilling  to  participate  because  of  the  lack  of  tangible 
benefits  derived  from  their  participation  in  the  EMCRO  project.  MFMC 
personnel  indicated  physicians  do  not  seem  interested  in  taking  part 

in  research  and  development  activities,  but  only  in  operational  programs 
where  their  practices  can  be  directly  benefited  because  of  input  and 
particpation.    This  ties  in  with  the  second  point  which  is: 

2.  The  lack  of  incentives  built  into  the  project  to  give  potential  parti- 
cipants some  immediate  benefits  for  use  of  their  information  and  time 
in  collecting  project  data.    The  complexities  of  the  multi-use  system 
contributed  to  this  factor  and  were  underestimated  by  MFMC  personnel. 

The  three  physicians  involved  in  the  multi-use  project  at  this  point  include 
an  internist  who  parti cpated  in  the  EMCRO  program  and  who  is  highly  committed  to 
the  multi-use  concept,  and  two  pediatricians  whose  future  participation  in  the 
project  is  questionable;  they  do  not  see  the  need  for  the  multi-use  system 
in  their  practicies.    At  this  point  MFMC's  estimation  is  that  500  physician- 
patient  encounters  have  been  collected  and  entered  in  the  multi-use  computerized 
system. 

The  present  multi-use  system  staff  includes  one  data  entry  person,  one  data 
analyst,  one  programmer,  and  an  assistant  executive  director  who  is  project 
manager.    MFMC  personnel  noted  during  the  February  site  visit  there  was  not 
enough  time  to  determine  whether  there  would  be  trouble  with  the  software  de- 
veloped by  Dikewood.    However,  a  flexible  language  had  been  used  and  problems 
were  not  anticipated.    It  was  also  noted  some  aggregate  data  reports  based  on 
the  encounters  of  the  three  physicians  involved  in  the  project  would  be  generated. 
The  present  grant  application  for  multi-use  expires  July,  1976,  although  MFMC 
will  process  claims  and  information  until  September,  1976,  with  a  no-cost  exten- 
sion from  BHSR. 

The  four  main  problem  areas  in  the  development  and  implementation  of  the 
multi-use  system  can  be  summarized  as  follows: 
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1.  Need  for  funding  for  further  research  in  support  of  multi-use 

2.  Need  for  increased  involvement  of  physicians  in  multi-use 

3.  Need  to  involve  physicians  committed  enough  to  learn  the  data  col- 
lection forms 

4.  Lack  of  control  over  development  of  software  of  an  outside  company 
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SECTION  III 


HCMS     STUDY  INFORMATION 


Two  visits  were  made  to  MFMC,  one  on  October  27,  1975,  the  other  on 
February  12,  1976.    The  first  visit  was  constrained  by  the  fact  that  HCMS 
had  to  interview  three  of  the  Multnomah  Foundation's  staff  people  at  the  same 
time,  making  it  difficult  to  record  the  discussion  on  paper. 

The  following  documents  were  received  from  MFMC: 

1.  The  Final  Report  of  the  Multnomah  Foundation  for  Medical  Care  Experi- 
mental Medical  Care  Review  Organization  (January,  1975) 

2.  Grant  Application  for  the  Multi-Use  Medical  Data  System  (February  1, 
1974) 

3.  Two  supplementary  reports  on  the  Multi-Use  Medical  Data  System 
(April  12,  1974,  and  May  4,  1974) 

4.  An  organizational  chart  of  Multnomah  Foundation  for  Medical  Care 

5.  Multi-use  data  forms,  most  of  which  were  dated  on  July  11,  1975 

6.  Examples  of  computer  displays  from  multi-use  project 

The  documentation  comprised  more  than  800  pages  and  provided  study  staff 
with  detailed  information  about  MFMC's  EMCRO  project  and  the  development  of  the 
multi-use  project. 

The  interviews  on  both  site  visits  were  conducted  with  the  Executive  Director 
of  MFMC  and  the  Associate  Executive  Director.  The  Project  Director  for  the  multi- 
use  project  was  also  interviewed  on  the  first  visit. 

The  following  topics  were  the  major  points  covered  during  the  on-site  inter- 
views : 

1.  Development  of  the  Multi-Use  Data  System 

2.  Development  of  Multi-Use  System  methodology 

3.  Data  collection  and  processing  for  multi-use 

4.  Physician  participation  in  multi-use  project 

5.  Problem  areas  in  multi-use  methodology 

6.  Interrelationship  between  EMCRO  results  and  multi-use  project 
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CONNECTICUT  AMBULATORY  CARE  STUDY 


SECTION  I 
INTRODUCTION 


The  Connecticut  Ambulatory  Care  Study  (CACS)  project  was  initiated  in 
1972  through  the  joint  efforts  of  representatives  of  Connecticut  State  Medical 
Society  (CSMC),  Yale  University,  the  University  of  Connecticut  (UC)  and  the 
Connecticut  Regional  Medical  Program  (CRMP).    Each  entity  has  expressed  some 
interest  in  studying  quality  assurance  methods  and  issues  within  solo  and 
group  practices  within  the  State  of  Connecticut.    The  contract  to  conduct  the 
research  was  negotiated  between  CSMC  and  CRMP,  and  started  officially  on 
October  5,  1972. 

This  description  does  not  attempt  to  develop  a  detailed  chronological  dis- 
cussion of  the  CACS  project,  but  will  only  present  important  research  components 
of  the  study.    Study  staff  noted  the  project  has  a  complicated  history:    as  a 
result  the  project  has  seemed  to  suffer  from  a  lack  of  consistent  direction 
complicated  by  funding  and  organizational  problems. 

The  goals  of  the  study  were: 

1.  To  evaluate  the  applicability  of  methods  for  patient  care  evaluation 
developed  in  outpatient  settings  in  Connecticut 

2.  To  develop  techniques  for  monitoring  the  quality  and  appropriateness 
of  ambulatory  care 

3.  To  conduct  a  limited  evaluation  of  quality  care  delivered  in  alter- 
nate settings,  including  a  minimum  of  four  hospital  outpatient 
departments 

Research  activities  focused  on  quality  assurance  methodology  and  patient 
evaluation  in  solo  practitioner  offices.    Data  were  collected  in  five  hospitals 
in  the  greater  Hartford  and  greater  New  Haven  areas  which  had  the  largest  popu- 
lations and  a  sample  of  solo  practitioners.    Information  was  obtained  from 
hospital  physicians  who  provided  care  to  hospital  patients  presenting  selected 
problems  in  a  two-week  period;  153  physicians  participated.    All  primary  care 
hospital  practitioners  participated  (i.e.,  general  practitioners,  internists, 
and  pediatricians).    Ultimately,  a  stratified  random  sample  of  155  solo  prac- 
tioners  was  selected  from  all  primary  care  physicians  licensed  in  the  greater 
Hartford  and  New  Haven  areas.    After  this  sample  was  contacted,  110  physicians 
indicated  their  willingness  to  participate. 

The  revised  objectives  of  the  study  were  to  identify  a  large  sample  of 
solo  practitioners  and  to  sample  patient  medical  records  within  their  offices. 


323 


The  performance  of  physicians  was  then  to  be  described  according  to  perfor- 
mance level  of  care  based  on  predetermined  criteria  developed  by  a  technical 
advisory  group  associated  with  the  CACS.    There  were  three  levels  of  analysis. 
First,  performance  measures  were  presented  by  practice  setting  and  major  specialty 
on  each  criterion  for  all  performance  factors.    Second,  differences  in  perfor- 
mance in  major  professional  and  organizational  characteristics  were  identified. 
Third,  significant  attributes  were  length  of  appointment,  degree  of  delegation 
of  responsibility  for  tests  and  procedures  to  nonphysician  staff,  number  of 
MDs  in  the  practice  (private  practice  only),  degree  to  which  physicians  are 
disturbed  by  "trivial  complaints,"  physician  income,  specialty,  length  of  time 
in  practice  and  many  others.    Other  analyses  provided  information  on  differences 
in  care  by  sociodemographic  characteristics  of  the  patient  population.  Descrip- 
tive analyses  of  physicians  and  their  practices  were  also  available.    The  object 
was  to  take  a  time  reference  sample  of  patients  seen  by  these  110  solo  practi- 
tioners and  audit  those  charts  for  specific  developed  criteria.    CACS  staff 
also  developed  a  scoring  system  which  would  compute  a  performance  score  for 
each  physician. 

The  CACS  program  was  included  in  the  study  to  reflect  operational  research 
activities  in  quality  assurance  techniques. 
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SECTION  II 


QUALITY     ASSURANCE  PROGRAM 


BACKGROUND  AND  RESEARCH  COMPONENTS 

Prior  to  CACS,  initiated  in  1972,  several  key  persons  involved  in  health 
services  research  had  expressed  an  interest  in  researching  quality  assurance 
and  had  begun  informally  communicating  with  prominent  members  of  the  Connec- 
ticut medical  community.    These  persons  began  to  bring  the  idea  to  representatives 
of  various  organizations.    After  the  CSMC,  Yale  University  and  University  of 
Connecticut  began  to  interact  with  CRMP,  it  was  determined  (early  in  1971) 
that  funding  might  be  obtained  from  the  national  Regional  Medical  Program  to 
initiate  the  study. 

A  contract  was  negotiated  with  the  state  medical  society,  which  had  over- 
all responsibility  for  implementation  of  the  research  design  and  the  structure 
of  the  program.    To  initiate  the  project,  two  advisory  committees  were  appointed: 
1)  a  General  Advisory  Committee  was  organized  by  the  medical  society  and  the 
investigators  to  ensure  scientific  conduct  of  the  project,  and  2)  a  Technical 
Committee  was  designated  by  the  advisory  committee  to  oversee  the  development 
of  criteria  for  all  selected  diseases  and  problem  areas. 

As  stated  in  the  final  contract,  specific  objectives  included:    (a)  evalua- 
tion of  the  applicability  of  methods  for  patient  care  evaluation  developed  in 
other  geographic  areas  and  settings,  (b)  development  of  standards  and  techniques 
for  monitoring  the  quality  of  ambulatory  care,  (c)  evaluation  of  the  effect  of 
alternative  organizational  devices  for  primary  health  care  delivery,  and  (d) 
construction  of  mechanisms  to  best  serve  the  educational  needs  of  practicing 
physicians. 

The  current  president  of  CSMC  was  designated  principal  investigator  and 
two  co-principal  investigators  were  appointed.    A  project  director  and  a  medical 
director  were  hired.    The  project  director  was  responsible  for  hiring  medical 
records  technicians,  abstractors  and  research  assistants.    The  record  techni- 
cians examined  the  quality  of  medical  records  and  review  records,  the  abstractors 
abstracted  medical  records  based  on  the  criteria  maps,  and  research  assistants 
conducted  interviews  and  filled  out  questionnaires. 

The  sample  selection  was  based  on  RMP  contractual  requirements.    The  other 
selection  process  involved  choosing  disease  conditions  to  be  reviewed.  This 
determination  was  made  through  interaction  among  the  two  principal  investi- 
gators, the  Technical  Committee,  the  General  Advisory  Committee  and  research 
staff  personnel.    The  Technical  Committee  generated  a  list  of  15  topics,  divided 
into  four  categories: 

A.    Chronic  Disease 

1.  Hypertension,  adult 

2.  Diabetes  mellitus,  adult 

3.  Arteriosclerotic  heart  disease,  adult 

4.  Asthma,  pediatric  only 
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B.  Acute  Disease 

5.  Urinary  tract  infection  or  dysuria,  adult  and  pediatric 

6.  Pharyngitis 

7.  Otitis  media,  pediatric  only 

C.  Problem  Areas 

8.  Chest  pain,  adult 

9.  Adult  and  pediatric  abdominal  pain 

10.  Adult  and  pediatric  obesity 

D.  Skills 

11.  Well -adult  physical  exam 

12.  Well -child  exam 

13.  Electrocardiogram  interpretation 

CACS  personnel  completed  the  task  of  choosing  topics  for  study  through  exten- 
sive discussions  of  each  category  and  topic.    Eventually,  several  topics  were 
dropped  from  the  initial  list,  including  electrocardiograms,  well-adult  exams, 
arteriosclerotic  heart  disease,  diabetes  mellitus,  asthma  and  obesity.  Docu- 
mentation notes  the  major  consideration  in  ruling  out  certain  topics  was  the 
difficulty  of  collecting  medical  record  information  for  the  topics,  due  to  low 
incidence  of  the  problem  or  high  cost  of  abstracting. 

After  the  final  topics  for  study  were  selected,  the  Technical  Committee 
divided  into  subcommittees  to  develop  criteria  for  the  topics.  Subcommittees 
were  organized  to  represent  different  specialties  and  practice  settings.  Docu- 
mentation notes  over  65  physicians  were  brought  in  to  assist  in  criteria  develop- 
ment.   Several  steps  in  this  process  were  outlined: 

1.  Each  physician  was  interviewed  by  CACS  staff  to  obtain  an  initial 
criteria  list  (interviews  were  audio  recorded) 

2.  Meetings  were  held  throughout  Connecticut  to  allow  the  consultant 
physicians  to  review  initial  criteria  lists 

3.  After  group  discussions  were  completed,  each  physician  was  asked 
to  rank  criteria  items 

Once  this  was  completed,  CACS  staff  compiled  frequencies  for  the  criteria 
items  and  computed  the  reliability  of  individual  physicians.    The  documentation 
suggested  criteria  development  involved  many  informal  interactions  between 
physicians  in  arriving  at  consensus  and  judgments.    CACS  staff  encouraged  phy- 
sicians to  develop  only  essential  criteria  items,  preferably  less  than  ten 
items  per  topic.    This  was  not  possible  and  criteria  items  eventually  ranged 
from  40  to  115  items.    Criteria  items  were  finally  selected  in  deliberations 
between  Technical  Committee  members  and  CACS  staff.    CACS  staff  indicated  that 
including  physicians  from  diverse  practice  settings  (e.g.,  hospital-based, 
academic,  solo  practice),  although  necessary,  caused  much  disagreement  in  the 
resolution  of  essential  criteria  items.    Throughout  the  study,  each  topic  chosen 
to  be  part  of  the  study  had  a  specific  panel  of  physicians  chaired  by  a  member 
chosen  by  staff.    This  panel  was  utilized  to  answer  all  CACS  questions  related 
to  that  topic,  as  well  as  to  update  and  revise  criteria  based  on  CACS  program 
operational  experiences.    A  criteria  checklist  was  also  filled  out  by  the 
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participating  physician  to  document  individually  held  criteria  and  to  validate 
criteria  development  by  the  physician  panels.    Participating  physicians  valida- 
ted panel  criteria  which  were  predetermined.    Observation  of  a  sample  of  pedia- 
tricians validated  the  measures  obtained  from  patient  records. 

CACS  data  collected  went  beyond  abstracting  data  from  medical  records. 
CACS  staff  developed  and  conducted  interview  schedules  with  the  participating 
private  and  hospital  practitioners.    CACS  personnel  indicated  a  professional 
leadership  questionnaire  was  administered  to  assess  a  practitioner's  personal 
assessment  of  his  own  managerial  style.    An  organizational  questionnaire  was 
administered  to  assess  the  organizational  style  of  private  and  hospital  prac- 
tices.   Questionnaires  were  developed  by  several  CACS  research  assistants.  A 
questionnaire  designed  to  obtain  information  on  the  physician's  personal  and 
professional  attributes  was  designed  and  administered.    CACS  staff  also  col- 
lected some  sociodemographic  data  (age,  address,  sex)  as  well  as  the  presenting 
problems  for  all  patients  appearing  in  the  two-week  log  period. 

Once  the  interviews  were  completed,  CACS  staff  focused  on  the  main  data 
collection  activity.    The  abstraction  of  medical  record  data  was  conducted  by 
several  CACS  research  assistants  trained  for  such  field  work.  Information 
from  sample  medical  records  was  recorded  via  a  criteria  abstract  package  which 
included  a  list  of  the  criteria,  instructions  for  their  implementation  and  an 
answer  sheet  in  which  data  were  recorded  by  field  workers.    The  CACS  program 
abstractors  went  through  a  one-  to  two-week  training  session.    The  training 
schedule  included  classroom  work,  lectures  and  visual  aid  presentations,  as 
well  as  onsite  visits  for  abstract  and  chart  testing.    One  person,  a  certified 
medical  records  technician,  was  responsible  for  data  accuracy  and  control.  CACS 
instituted  various  validity  checks  of  the  abstracting  process  based  on  monitoring 
50%  of  abstracted  medical  records.    Abstractors  were  instructed  to  collect  certain 
data  elements  on  disease  entities  which  then  could  be  compared  with  criteria. 

Office  medical  records  served  as  the  data  source  on  each  practitioner. 
Records  were  chosen  by  asking  office  personnel  to  compile  a  two-week  log  of 
patient  visits  for  the  identified  topics.    Once  the  proper  sample  number  was 
available  in  each  office,  CACS  staff  arranged  to  make  on-site  visits. 

CACS  staff  spent  considerable  time  coordinating  the  field  work.  Initial 
visits  were  made  to  all  sites  to  negotiate  with  participant  staffs,  to  assess 
the  quality  of  medical  records  to  be  abstracted  and  to  determine  the  incidence 
of  topics  to  be  reviewed.    The  data  analysis  conducted  by  CACS  to  describe  phy- 
sician performance  in  accordance  with  criteria  was  constructed  around  a  trilevel 
scoring  system: 

1.  System  Zero,  where  each  criterion  is  scored  either  zero  (0)  or  (1), 
depending  on  whether  evidence  existed  that  a  criterion  was  met. 

2.  System  One  assigned  relative  values  to  each  criterion  item  "by  taking 
the  average  weight  given  to  each  item  by  a  panel  of  physician  experts." 

3.  System  Two  requires  each  criterion  to  be  assessed  according  to  all 
other  information  available,  based  on  CACS  physicians'  suggestion 
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that  more  characteristics  of  patients  and  physicians  be  taken  into 
account.  The  scoring  required  the  reviewer  to  assign  a  range  from 
+3  (highest)  to  -3  (lowest)  for  specific  criteria. 

Because  of  the  large  number  of  criteria  for  each  topic  on  which  physician 
performance  could  be  measured  (there  could  be  between  40  and  115  criteria  per 
topic),  CACS  used  factor  analysis  to  reduce  the  larger  set  of  dependent  variables 
(criteria)  into  smaller  sets.    Reducing  the  number  of  dependent  variables  sub- 
sequently related  to  the  study's  independent  variables,  aided  in  interpreting 
results.    Reduction  was  accomplished  by  finding  subgroups  of  criteria  in  each 
topic  which  were  related  to,  yet  uncorrelated  with,  the  criteria  found  in 
other  subgroups.    Factor  analysis  was  used  to  identify  these  subgroupings  of 
criteria.    In  fact,  multiple  factor  analyses  were  performed  in  each  topic,  and 
the  identification  of  factors  was  based  on  a  combination  of  statistical  and  judg- 
mental concerns.    Generally,  criteria  which  loaded  (i.e.,  correlated  with)  a 
factor  at  a  level  of  0.40  or  greater  were  considered  important  in  defining  that 
factor;  the  criteria  statistically  associated  with  the  factors  had  to  be  inter- 
related according  to  meaning  (content)  before  they  were  accepted  as  defining 
a  factor.    Several  reliability  and  validity  checks  were  performed  for  each 
factor  within  the  topic  areas. 

Having  performed  the  factor  analyses  and  established  the  factors,  a  per- 
formance score  on  each  factor  was  computed  based  on  the  compliance  or  noncom- 
pliance of  physicians  with  the  various  criteria  comprising  each  factor.  A 
single  score  for  performance  was  computed  for  each  subgroup  of  criteria  which 
emerged  from  the  factor  analysis,  and  these  scores  were  then  related  to  the 
independent  variables  of  the  study.    In  the  CACS  scheme,  all  criteria  items 
associated  with  a  factor  received  an  equal  weight  in  the  calculation  of  a  per- 
formance score  for  each  factor  (i.e.,  they  all  received  a  weight  of  one). 
Criteria  found  to  be  unrelated  to  a  factor  did  not  contribute  to  the  performance 
score  calculated  for  that  factor  (i.e.,  those  items  received  a  weight  of  zero). 

CACS  reported  that  this  analysis  was  conducted  for  all  three  scoring 
systems,  comparing  each  system  for  the  difference  in  factors  yielded  through 
the  correlation  analysis.    Documentation  notes  that  there  was  "a  great  deal 
of  consistency  across  (scoring)  systems,"  that  is,  similar  factors  were  identi- 
fied across  systems  despite  conceptual  differences  in  scoring. 


ADDITIONAL  COMMENTS 

During  interviews,  CACS  personnel  identified  several  problems  which  af- 
fected the  project  and  would  probably  affect  any  project  with  similar  objectives. 
These  problem  areas  included: 

1.  The  diversity  in  solo  practitioner  office  settings  and  geographical 
locations  of  solo  practitioners 

2.  The  validity  of  medical  records  in  terms  of  measuring  solo  practi- 
tioner performance  levels 

3.  The  cost  of  abstracting  and  data  collecting 

4.  The  cost  of  administering  several  questionnaires 
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SECTION  III 

HCMS     STUDY  INFORMATION 


CACS  staff  was  initially  contacted  on  September  24,  1975,  regarding  parti- 
cipation in  this  study.    The  importance  of  CACS's  project  was  communicated  to 
study  staff  by  Bureau  of  Quality  Assurance  personnel.    Study  staff  requested 
CACS  documentation  which  described  their  program  and  the  following  documents 
were  received  and  compiled: 

1.  Copy  of  contract  between  the  Connecticut  RMP  and  Connecticut  State 
Medical  Society  to  conduct  the  CACS  (1972) 

2.  Correspondence  from  the  CACS  Research  Director  to  the  Bureau  of 
Quality  Assurance  detailing  project  activities  and  plans  (May  2,  1975) 

3.  Abstract  forms  for  all  the  disease  entities  (May-August,  1974) 

4.  CACS  Progress  Report  (October  10,  1975) 

Significant  amounts  of  information  were  collected  during  the  site  visit  on 
October  16,  1975.    Several  hours  were  spent  with  CACS  personnel  such  as  the 
project  director  and  the  research  assistant  (who  developed  and  administered 
the  Physician  Characteristics  Interview  to  all  participating  private  and  hos- 
pital practitioners). 

Study  staff  interviewed  another  senior  research  assistant  who  developed 
the  method  of  reducing  performance  data  and  constructed  the  factor  analysis. 
(Information  was  computerized  by  programmers.)    At  this  time,  results  of  CACS 
research  activities  concerning  the  impact  of  quality  assurance  on  practices 
and  assessment  by  the  solo  practitioners  of  findings  is  being  reviewed  by  the 
state  medical  society  and  will  be  released  at  the  end  of  this  year  (1976). 
CACS  staff  is  now  developing  a  strategy  for  obtaining  practitioner  feedback. 
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UNIVERSITY  OF  MINNESOTA 


Department  of  Physical  Medicine  and  Rehabilitation 


SECTION  I 
INTRODUCTION 


The  Department  of  Physical  Medicine  and  Rehabilitation  (DPM&R) 
is  a  large  outpatient-inpatient  department  in  the  University  of 
Minnesota  Medical  School  complex.    DPM&R  is  divided  into  three  major 
service  centers  including  (1)  Clinical  Services,  (2)  Research,  and 
(3)  Education.    Under  the  supervision  of  the  medical  director,  who 
is  also  head  of  the  department,  Clinical  Services  is  responsible  for 
care  of  approximately  4,000  patients  per  year,  both  ambulatory  and 
inpatient  visits.    DPM&R  is  the  only  department  in  the  medical  center 
currently  implementing  a  formalized  quality  assurance  program.  The 
department  was  established  in  1947.    About  1972,  the  department,  through 
its  Research  and  Training  Center  and  by  means  of  a  grant  from  the  Social 
Rehabilitation  Service  (SRS,  DHEW)  began  an  evaluation,  in  conjunction 
with  INTERSTUDY,  of  the  quality  of  rehabilitation  services  using  an  out- 
come assessment  methodology.    This  study,  which  focused  on  outcomes  of 
stroke  patients  who  had  experienced  rehabilitation,  was  based  on 
John  Williamson's  Health  Accounting  program,  developed  at  the  Johns 
Hopkins  University.    At  the  completion  of  that  study,  DPM&R  became 
involved  in  the  present  Williamson  Health  Accounting  Project  (July, 
1974  to  June,  1977).    The  subject  of  this  description  is  the  specific 
objections  of  the  latter  project  as  outlined  in  Williamson's  first 
annual  progress  report: 

1.  Consolidation  and  analysis  of  previous  health  accounting 
experiences  and  development  of  new  project  resources 

2.  Evaluation  of  the  reliability  and  validity  of  the  basic 
procedures  of  the  health  accounting  method 

3.  Examination  of  the  impact  of  health  accounting  on 
patient  health 

4.  Evaluation  of  the  impact  of  health  accounting  on  clinic 
organization  and  administration 

It  should  be  noted  that  DPM&R  is  one  of  six  clinics  in  the  current 
Health  Accounting  Project. 
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SECTION  II 


QUALITY     ASSURANCE  PROGRAM 


This  section  will  present  the  elements  of  DPM&R's  quality  assurance 
program  in  three  sections: 

1.  Background 

2.  Planned  program  methodology 

3.  Specific  operational  activities 

Planned  program  methodology  is  described  based  on  the  documents 
provided  by  Dr.  Williamson  and  DPM&R.    Information  for  operational 
activities  was  obtained  through  the  onsite  interviews. 


BACKGROUND 

The  DPM&R  is  involved  in  the  national  Health  Accounting  Project. 
As  indicated,  development  of  the  current  health  accounting  study  began 
with  the  stroke  study  about  1972,  through  which  the  project  director 
became  acquainted  with  Dr.  Williamson  and  his  health  accounting  metho- 
dology.   After  consulting  with  Dr.  Williamson  throughout  the  stroke  study, 
the  project  director  agreed  in  1974  to  participate  with  him  in  the  current 
project.    Because  the  health  accounting  methodology  had  not  previously 
been  assessed  in  a  setting  like  the  DPM&R's,  Dr.  Williamson  was  eager  to 
test  its  effectiveness  in  such  a  setting.    Plans  for  the  project  were  begun 
in  September,  1974.    The  project  director  stated  the  reasons  for  continuing 
this  type  of  quality  assurance  activity  in  DPM&R  are  (1)  the  department  was 
willing  to  test  the  methodology,  (2)  the  project  had  independent  funding, 
and  (3)  the  department  had  experience  with  outcome  studies. 


PLANNED  PROGRAM  ACTIVITIES 

To  initiate  the  Health  Accounting  methodology,  a  clinic  must  take 
four  steps: 

1.  Organize  a  Quality  Assurance  Board 

2.  Assign  a  physician  coordinator 

3.  Hire  a  health  accountant 

4.  Recruit  members  for  priority  topic  and  study  design  teams 

It  is  the  duty  of  the  Quality  Assurance  Board  (QAB)  to  oversee  the 
project  within  the  clinic.    This  group  typically  consists  of  seven 
members  representing  administrative  personnel,  medical  specialties,  the 
Board  of  Trustees,  and  if  possible,  consumers.    This  group  is  required 
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to  meet  three  times  a  year  for  the  duration  of  the  program.  The 
general  purpose  of  this  board  is  to  approve  team  membership  for 
the  priority  topic  team  and  study  design  teams;  further,  the  QAB 
makes  assessment  recommendations  and  institutes  corrective  action 
in  the  cl inic. 

The  physician  coordinator  acts  as  a  liaison  between  the  QAB 
and  the  various  study  teams  and  provides  continuity  through  all 
stages  of  the  project. 

The  health  accountant  is  a  nonmedical  person  hired  to  conduct 
the  actual  quality  assurance  studies  (mainly  data  collection  and 
compilation).    It  is  the  health  accountant  who  has  the  responsibility 
of  gathering  all  necessary  data  to  evaluate  outcomes  either  by  record 
review  or  patient  interview. 

Two  topic  teams,  composed  of  seven  to  nine  members,  are  responsi- 
ble for  developing  separate  lists  of  priority  study  topics  to  be 
completed  during  the  study  period.    These  topics  are  generated  by 
using  small  group  estimation  techniques  to  identify  "Achievable 
Benefits  Not  Achieved"  (ABNA).    The  team  members  rate  the  final  topics 
according  to  optimum  achievable  benefits  under  the  constraints  of 
present  clinic  practice  conditions.    These  are  then  the  priorities  for 
the  study  topics. 

Study  teams  are  then  organized  to  investigate  each  ABNA  topic. 
A  study  team  usually  consists  of  approximately  five  members  chosen  on 
the  basis  of  knowledge  and  experience  with  the  study  topics.  This 
team  has  the  responsibility  for  confirming  the  decision  of  the  priority 
team  and  designing  the  study  to  be  implemented  by  the  health  accountant. 
The  study  team  delineates  the  patient  population  to  be  sampled,  the 
sampling  procedures  and  the  measurements  to  be  applied  in  assessing  the 
acceptable  outcome  standards  which  they  set.    Finally,  they  compare  the 
measured  results  of  the  standards  and  make  recommendations  to  the  QAB. 

The  key  concept  of  the  priority  setting  mechanism  is  ABNA.  ABNA 
refers  to  a  medical  care  benefit  which  is  presently  attainable,  but 
has  not  been  attained.    In  setting  study  priorities,  those  planning 
quality  assurance  programs  use  the  ABNA  framework  to  make  explicit  assump- 
tions regarding  how  much  and  what  type  of  care  benefit  is  possible  within 
the  constraints  of  present  scientific  and  medical  knowledge.    It  also 
forces  quality  assurance  planners  to  make  explicit  assumptions  regarding 
what  values  are  to  be  applied  in  defining  benefits,  and  in  determining 
how  much  of  what  type  of  medical  care  warrants  the  most  immediate 
attention. 

The  estimation  of  ABNA  is  made  practical,  according  to  Williamson, 
by  the  development  of  what  he  calls  "small  group  estimation  technology." 
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This  procedure  centers  around  the  staff's  estimate  of  achievable 
benefits  not  achieved,  followed  by  a  discussion  of  the  first  round 
of  estimates.    A  second  series  of  estimations  is  made  and  a  final 
group  estimate  of  ABNA  is  generated.    The  quantification  of  the 
outcome  measure  involves  not  only  an  estimate  of  ABNA,  but  also  an 
estimate  of  achievable  therapeutic  benefits. 

Both  these  measures  are  quantifiable  by  use  of  a  functional 
impairment  scale.    This  scale  is  an  ordinal  measurement  of  the 
degree  of  impairment  in  life  activity  for  the  population  at  risk. 
Documents  define  these  levels  of  functional  status:    (1)  no 
impairment    (2)  asymptomatic  with  detectable  impairment    (3)  symp- 
tomatic at  major  life  activity    (4)  restricted  from  major  life 
activity    (5)  dependent  on  others  for  major  life  functions  and 
(6)  death.    Using  these  techniques  the  two  priority  teams  validate 
each  other's  lists  of  high  and  low  ABNA  topics.    These  topics  are 
then  assigned,  various  study  teams  determine  formal  confirmation, 
and  investigation  of  the  particular  topic  begins. 

Williamson  outlines  the  duties  of  the  coordinator,  health 
accountant,  and  priority  and  study  teams  within  the  five-stage 
framework  of  health  accounting  as  described  below: 

Stage  1  activities  require  the  priority  team,  under  the  direc- 
tion of  the  coordinator,  to  identify  topics,  which,  if  studied,  will 
probably  result  in  the  most  patient  health  improvement. 

Stage  2  consists  of  designing  and  implementing  an  initial  out- 
come assessment  of  a  high  priority  topic.    The  study  team  designs 
the  entire  study,  but  principally  sets  outcome  standards  and  measures. 
The  information  for  assessments  is  gathered  by  the  health  accountant 
under  the  supervision  of  the  physician  coordinator.    If  the  health 
outcomes  are  within  the  standards  established  by  the  clinic  staff, 
further  study  of  that  topic  is  not  warranted.    On  the  other  hand, 
if  measured  outcomes  do  not  meet  standards,  more  definitive  study 
is  required. 

The  study  team,  coordinator,  and  health  accountant  perform 
similar  assessment  roles  in  Stages  3,  4,  and  5.    Stage  3  consists 
of  a  more  definitive  inquiry  to  confirm  that  outcomes  do  not  meet 
clinic  standards,  and  to  identify  the  determinants  of  inadequate 
outcomes.    Stage  4  consists  of  planning  and  implementing  an  effort 
to  correct  the  deficiencies  identified.    Stage  5  is  a  repetition  of 
the  initial  outcome  assessment  to  determine  whether  the  improvement 
effort  was  successful.    If  standards  are  met,  the  project  is 
completed  and  a  monitoring  schedule  is  established  for  periodic 
reassessment.    If  acceptable  improvement  is  not  achieved,  then 
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stages  3  and  4  are  repeated  until  standards  are  met  or  the  staff 
determines  that  the  possible  gain  does  not  seem  worth  the  effort. 
Figure  1  summarizes  these  stages. 
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Figure  1:    Stages  of  the  Health  Accounting  Recycle  Project  in 
its  strategy  to  assure  an  improved  outcome.  "From  Health  Accounting": 
An  Outcome  Based  System  of  Quality  Assurance  Illustrations  Applicable 
to  Hypertension,  J.W.  Williamson,  John  Hopkins  University. 


OPERATIONAL  ACTIVITIES 

As  of  the  site  visit  date,  the  DPM&R  Health  Accountant  Program 
had  completed  the  study  of  urinary  tract  infection  (UTI)  in  spinal 
cord  injury  patients.    The  department  is  also  in  the  process  of  con- 
ducting other  studies  in  chronic  low  back  pain  and  use  of  intra- 
muscular neurolysis  to  lessen  spasticity.    At  the  time  of  the  site 
visit,  the  chronic  low  back  pain  study  was  in  stage  5  and  the  intra- 
muscular neurolysis  study  was  in  stage  1.    The  completed  study  on  UTI 
will  be  used  as  an  example  to  describe  the  operational  status  of 
health  accounting  in  the  DPM&R. 

It  should  be  noted  that  DPM&R  operational  activities  closely 
correlate  with  methodological  plans  described  in  the  preceding  section. 

Prior  to  the  first  meeting  of  the  study  team,  the  priority  teams 
had  identified  UTI  in  spinal  cord  injury  patients  as  the  most  appro- 
priate topic  to  be  addressed.    After  turning  this  problem  over  to  the 
study  team,  discussions  began  on  what  operational  definitions  would  be 
used  to  delineate  the  study  population.    No  formal  definitions  were 
decided  during  this  meeting  (June,  1975)  and  it  was  agreed  that  the 
estimates  of  benefits  would  be  based  on  a  general  nonspecific  definition 
of  the  health  topic. 
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The  team  then  made  first  (later  revised)  estimates  of  the 
maximum  acceptable  percent  of  patients  at  each  level  of  the  func- 
tional limitation  scale.    To  determine  if  a  therapeutic  outcome 
assessment  would  reveal  a  significant  difference  between  desirable 
and  actual  outcomes  in  the  population,  team  made  estimates  of  the 
percentage  of  patients  at  each  level  of  the  functional  limitation 
scale  and  of  the  percent  of  patients  "out  of  control"  for  the 
problem.    Certain  modifications  were  made  on  the  six-point  func- 
tional limitation  scale  developed  by  Williamson,  to  make  it  more 
applicable  to  the  study  at  hand.    Other  modifications  were  incor- 
porated later  as  the  topic  became  clearer,  though  the  modifications 
did  not  neglect  the  substantive  aspects  of  the  methodology. 

The  difference  between  the  estimated  level  of  impairment  for 
the  out-of-control  group  and  the  estimated  level  of  acceptable 
impairment  was  used  to  express  the  amount  of  improvement  that  the 
team  believed  could  be  accomplished  by  changing  the  care  given  to 
patients  with  UTIs.    The  team  then  discussed  various  ways  of 
collecting  the  appropriate  data  to  verify  these  estimates.  Next, 
discussions  were  held  to  determine  what  type  of  corrective  action 
could  be  taken  to  bring  the  level  of  care  now  being  achieved  up  to 
the  acceptable  level  identified  (given  the  estimates  were  an  accurate 
reflection  of  the  problem). 

The  second  study  team  meeting  (July,  1975)  clarified  the  topic 
to  be  studied  in  more  exact  terms  (bacteriuria  was  chosen  as  the  UTI 
indicator).    The  functional  limitation  scale  was  then  finalized  to 
accurately  reflect  both  medical  and  social  function  limitations  to 
determine  the  distribution  of  the  study  population  among  the  cate- 
gories in  the  scale.    Data  collection  procedures  were  devised  to 
generate  information  which  could  be  used  to  classify  the  study  sample 
into  categories  of  functional  limitation.    These  data  involved  both 
clinical  laboratory  tests  and  social  function  (questionnaire) 
information. 

In  the  third  meeting  (November,  1975),  the  study  team  compared 
the  results  of  the  study  information  on:    1)  the  maximum  acceptable 
standards  for  each  level  of  the  functional  impairment  scale  which 
they  had  set  earlier,    2)  estimates  of  the  expected  results,  and 
3)  the  actual  outcomes  as  indicated  from  the  data  collected  on  the 
study  sample.    (No  figures  were  provided  on  the  size  of  the  sample). 
After  looking  at  the  results,  it  was  decided  that  it  would  be 
necessary  to  review  the  data  thoroughly  and  meet  again  in  two  weeks 
to  generate  decisions  for  improvement  and  other  action. 

The  final  meeting  (December,  1975)  of  the  study  team  for  UTI 
involved  a  review  and  discussion  of  the  discrepancies  identified  in 
the  results.    From  this  discussion,  certain  suggestions,  recommen- 
dations, and  guidelines  for  improvement  action  were  forwarded  to  the 
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DPM&R  Quality  Assurance  Board  (consisting  of  14  members)  for  review 
and  consideration.    At  this  point,  the  UTI  study  was  considered 
complete  and  the  next  ABNA  topic  (chronic  low  back  pain)  began  its 
cycle  with  a  new  study  team. 

It  should  be  emphasized  that  the  health  accountant  was  the 
person  responsible  for  carrying  out  the  actual  study  described  above. 
This  involved  questionnaire  construction  (in  conjunction  with  depart- 
ment physicians),  data  collection,  and  presentation  of  results.  The 
study  team  described  above  functioned  only  as  a  policy  setting  group 
for  the  study,  and  was  not  involved  in  the  actual  implementation  of 
the  study  methodology. 


ADDITIONAL  COMMENTS 

During  interviews,  DPM&R  personnel  mentioned  several  problems, 
in  the  context  of  their  setting,  related  to  administering  the  health 
accounting  methodology.    One  problem  identified  concerned  the  amount 
of  time  required  to  measure  outcomes  for  chronically  ill  patients. 
Since  this  was  the  second  time  the  health  accounting  methodology  had 
been  formally  implemented  in  a  rehabilitation  facility,  these  types 
of  questions  were  anticipated.    This  resulted  in  some  modifications 
in  the  implementation  of  the  health  accountant  methodology  noted 
earl ier. 

Another  problem  involved  the  demand  being  placed  on  the  depart- 
ment in  terms  of  patient  care,  teaching  responsibilities,  resident 
physician  supervision  and  research  functions.    It  was  indicated  that 
the  funding  level  for  the  project  was  not  adequate  to  support  a 
full-time  health  accountant.    However,  the  administrative  assistant 
for  the  department  indicated  the  health  accounting  program  would  be 
continued  after  the  National  Health  Accountant  Project  funding  ended 
and  estimated  the  cost  at  $16,500. 

The  DPM&R,  as  a  department  within  a  very  large  university  setting, 
is  somewhat  atypical  of  most  sites  in  this  survey.    The  organizational 
affiliations  and  administrative  processes  are  much  more  complex  than 
free-standing  facilities.    In  addition,  the  DPM&R  is  simultaneously 
involved  in  both  ambulatory  and  inpatient  quality  review. 

There  seems  to  be  good  support  from  the  entire  department  for 
the  quality  activities.    It  was  pointed  out  that  these  activities 
were  not  seen  as  having  only  research  functions,  but  also  as  having 
a  major  impact  on  the  type  of  health  services  delivered  by  the 
department  as  a  whole. 

The  medical  coordinator  of  the  quality  assurance  activities 
seems  to  provide  focus  and  direction  to  the  entire  project.  Because 
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of  his  previous  working  relationship  with  Dr.  Williamson,  and  his 
personal  commitment  to  the  health  accounting  approach,  he  has  been 
able  to  bring  enthusiasm  and  involvement  to  the  project.  The 
health  accountant  is  not  a  member  of  the  department.    He  works 
only  on  the  health  accounting  project,  and  indicated  this  causes 
some  difficulties  in  terms  of  arranging  physician  meetings,  an 
important  administrative  demand  of  the  health  accountant  method. 
Such  an  arrangement  requires  the  medical  coordinator  to  establish 
and  set  up  meetings  required  for  priority  setting  and  the  study 
teams.    There  is  little  discussion  of  the  health  accounting  project 
in  normal  staff  meetings,  although  there  is  a  good  interchange  among 
staff  members  when  they  meet  for  quality  assessment  purposes. 
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Revenue  sources  from  which  patient  services  are  financed  [for  1974) 
are  as  follows: 


No.  patients 

Percent  patients 

County 

66 

1.5 

Fee  for  Services 

277 

6.5 

Medicare/Medicaid 

1057 

25.0 

Othpr  "3rrl  Partv 
u  unci    ji  u   rcii  l y 

DO 

bo .  U 

Training  and  Research 

37 

.9 

Special  Funds 

215 

5.0 

Special  Services 

6 

.1 

Other 

0 

0.0 

Unknown 

134 

3.0 

Total 

4258 

100.0 
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SECTION  III 


DELIVERY     SYSTEM  CHARACTERISTICS 


The  DPM&R's  clinical  services  division  is  divided  into  four 
subcategories:    1)  Rehabilitation  Ward,    2)  Outpatient  Clinic 
3)  Rehabilitation  Center,  and    4)  Inpatient  Consultation.  Services 
administered  by  the  Rehabilitation  Center  include: 

1.  Physical  therapy 

2.  Speech  therapy 

3.  Occupational  therapy 

4.  Social  service  section 

5.  Psychology  section 

6.  Vocational  section 

7.  Psychiatric  consultation 

The  patient  population  characteristics  for  the  DPM&R  in  1974  may 
be  broken  down  as  follows: 


No.  patients 


No.  visits 


Inpatient 
Outpatient 


2712 
917 


34630 
11352 


Total 


3629 


45982 


Age 


No.  patients  admitted 


Over  64 
Unknown 


0-6 
6-11 
12-20 
21-40 
61-64 


194 
738 
317 
1141 
1157 
491 
220 


Total 


4258 
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SECTION  IV 

HCMS     STUDY  INFORMATION 


John  Williamson,  M.D.  of  John  Hopkins  University,  was  contacted 
in  October  of  1975  and  asked  if  he  could  suggest  any  ambulatory 
settings  with  operational  quality  assurance  programs  which  would  be 
interested  in  participating  in  the  study.    Dr.  Williamson  replied 
on  October  10,  1975,  indicating  he  would  welcome  such  an  opportunity 
and  suggested  consideration  be  given  to  some  or  all  of  the  six  set- 
tings in  his  Health  Accounting  Project.    DPM&R  was  chosen  because  it 
provided  an  opportunity  to  examine  a  quality  assurance  method 
implemented  in  a  large  department,  as  opposed  to  a  free  standing 
setting. 

The  medical  coordinator  for  the  Health  Accounting  Project  at 
the  University  of  Minnesota  DPM&R  was  contacted  on  October  21,  1975, 
to  ascertain  interest  in  participating.    In  a  letter  received 
January  2,  1976,  DPM&R  formally  agreed  to  participate  in  the  study. 
Initial  documentation  for  the  health  accounting  approach  had  been 
received  from  Dr.  Williamson,  which  outlined  the  methodology  being 
employed  at  DPM&R.    Information  about  DPM&R  activities  was  obtained 
during  the  site  visit  January  26  or  27,  1976,  at  which  time  the  Data 
Collection  Instrument  was  administered  and  further  DPM&R  documentation 
was  obtained.    All  documents  received  and  compiled  included: 

Documentation  provided  by  Williamson: 

1.  Annual  Progress  Report  Evaluating  an  Outcome  Based  Quality 
Assurance  Program  July  1,  1974  -  June  30,  1975  and  Appendices 

2.  Preliminary  Evaluation  of  Health  Accounting:  Content  Analysis 

3.  Health  Accounting:    An  Introduction 

4.  Understanding  ABNA 

5.  Health  Accounting  Training  Study 

6.  Evaluating  the  Impact  of  Health  Accounting:    The  Pretest 
Questionnaire 

7.  Health  Accounting:    An  Outcome  Based  System  of  Quality 
Assurance:  Application  to  Hypertension 

Documentation  proved  by  DPM&R: 

8.  Organizational  Chart  for  DPM&R 

9.  DPM&R  Patient  Information  System  Tabulations  for  1974 

10.  Minutes  for  Study  Design  Meetings  dated  June  19,  1975,  July  1,  1975, 
November  25,  1975,  December  9,  1975,  January  8,  1976,  undated  minutes 
following  January  8,  1976 

11.  Interview  Schedules  for  High  ABNA  Topic  #1 

12.  Health  Accounting  Project  Progress  Report  dated  November  10,  1975 

13.  Evaluating  Quality  of  Rehabilitation  Using  Outcomes  Assessment 

14.  A  Stroke  Study  done  by  the  DPM&R,  undated 
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These  documents  were  used  extensively  in  constructing  this  descrip- 
tion of  the  DPM&R's  Health  Accounting  Project.    Additional  information 
was  obtained  on  site  in  a  series  of  interviews  with  the  following 
personnel : 

1.  Medical  Coordinator  for  the  DPM&R  Health  Accounting 
Project 

2.  Health  Accountant 

3.  Previous  Health  Accountant 

4.  Administrative  Assistant 

5.  Head  of  the  Department 

A  total  of  eight  hours  of  interview  time  was  spent  with  DPM&R 
personnel.    Interviews  with  the  medical  coordinator,  the  health 
accountant  and  the  administrative  assistant  were  audio  recorded. 

Topics  covered  during  the  interviews  included: 

1.  The  history  of  the  Health  Accounting  Project  in  DPM&R 

2.  The  objectives  of  the  project 

3.  Unique  characteristics  of  DPM&R  which  affect  the  quality 
review  program 

4.  DPM&R  organization  for  health  accounting 

5.  Topics  under  study 

6.  Criteria  development 

7.  Cost  estimates 


/ 
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